summaryrefslogtreecommitdiff
path: root/drivers/crypto/hisilicon/sgl.c
AgeCommit message (Collapse)Author
2025-06-23crypto: hisilicon - Use fine grained DMA mapping directionZenghui Yu
The following splat was triggered when booting the kernel built with arm64's defconfig + CRYPTO_SELFTESTS + DMA_API_DEBUG. ------------[ cut here ]------------ DMA-API: hisi_sec2 0000:75:00.0: cacheline tracking EEXIST, overlapping mappings aren't supported WARNING: CPU: 24 PID: 1273 at kernel/dma/debug.c:596 add_dma_entry+0x248/0x308 Call trace: add_dma_entry+0x248/0x308 (P) debug_dma_map_sg+0x208/0x3e4 __dma_map_sg_attrs+0xbc/0x118 dma_map_sg_attrs+0x10/0x24 hisi_acc_sg_buf_map_to_hw_sgl+0x80/0x218 [hisi_qm] sec_cipher_map+0xc4/0x338 [hisi_sec2] sec_aead_sgl_map+0x18/0x24 [hisi_sec2] sec_process+0xb8/0x36c [hisi_sec2] sec_aead_crypto+0xe4/0x264 [hisi_sec2] sec_aead_encrypt+0x14/0x20 [hisi_sec2] crypto_aead_encrypt+0x24/0x38 test_aead_vec_cfg+0x480/0x7e4 test_aead_vec+0x84/0x1b8 alg_test_aead+0xc0/0x498 alg_test.part.0+0x518/0x524 alg_test+0x20/0x64 cryptomgr_test+0x24/0x44 kthread+0x130/0x1fc ret_from_fork+0x10/0x20 ---[ end trace 0000000000000000 ]--- DMA-API: Mapped at: debug_dma_map_sg+0x234/0x3e4 __dma_map_sg_attrs+0xbc/0x118 dma_map_sg_attrs+0x10/0x24 hisi_acc_sg_buf_map_to_hw_sgl+0x80/0x218 [hisi_qm] sec_cipher_map+0xc4/0x338 [hisi_sec2] This occurs in selftests where the input and the output scatterlist point to the same underlying memory (e.g., when tested with INPLACE_TWO_SGLISTS mode). The problem is that the hisi_sec2 driver maps these two different scatterlists using the DMA_BIDIRECTIONAL flag which leads to overlapped write mappings which are not supported by the DMA layer. Fix it by using the fine grained and correct DMA mapping directions. While at it, switch the DMA directions used by the hisi_zip driver too. Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Longfang Liu <liulongfang@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-09-06crypto: hisilicon - fix missed error branchYang Shen
If an error occurs in the process after the SGL is mapped successfully, it need to unmap the SGL. Otherwise, memory problems may occur. Signed-off-by: Yang Shen <shenyang39@huawei.com> Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-12crypto: hisilicon/sgl - Delete redundant parameter verificationChenghai Huang
The input parameter check in acc_get_sgl is redundant. The caller has been verified once. When the check is performed for multiple times, the performance deteriorates. So the redundant parameter verification is deleted, and the index verification is changed to the module entry function for verification. Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-01-10Merge tag 'v6.8-p1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto updates from Herbert Xu: "API: - Add incremental lskcipher/skcipher processing Algorithms: - Remove SHA1 from drbg - Remove CFB and OFB Drivers: - Add comp high perf mode configuration in hisilicon/zip - Add support for 420xx devices in qat - Add IAA Compression Accelerator driver" * tag 'v6.8-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (172 commits) crypto: iaa - Account for cpu-less numa nodes crypto: scomp - fix req->dst buffer overflow crypto: sahara - add support for crypto_engine crypto: sahara - remove error message for bad aes request size crypto: sahara - remove unnecessary NULL assignments crypto: sahara - remove 'active' flag from sahara_aes_reqctx struct crypto: sahara - use dev_err_probe() crypto: sahara - use devm_clk_get_enabled() crypto: sahara - use BIT() macro crypto: sahara - clean up macro indentation crypto: sahara - do not resize req->src when doing hash operations crypto: sahara - fix processing hash requests with req->nbytes < sg->length crypto: sahara - improve error handling in sahara_sha_process() crypto: sahara - fix wait_for_completion_timeout() error handling crypto: sahara - fix ahash reqsize crypto: sahara - handle zero-length aes requests crypto: skcipher - remove excess kerneldoc members crypto: shash - remove excess kerneldoc members crypto: qat - generate dynamically arbiter mappings crypto: qat - add support for ring pair level telemetry ...
2024-01-08mm, treewide: rename MAX_ORDER to MAX_PAGE_ORDERKirill A. Shutemov
commit 23baf831a32c ("mm, treewide: redefine MAX_ORDER sanely") has changed the definition of MAX_ORDER to be inclusive. This has caused issues with code that was not yet upstream and depended on the previous definition. To draw attention to the altered meaning of the define, rename MAX_ORDER to MAX_PAGE_ORDER. Link: https://lkml.kernel.org/r/20231228144704.14033-2-kirill.shutemov@linux.intel.com Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-12-01crypto: hisilicon/sgl - small cleanups for sgl.cWeili Qian
1. Remove unnecessary brackets in function hisi_acc_create_sgl_pool(). 2. Modify local variable type, ensure that the variable type is consistent with the variable type to be compared. 3. Because the function clear_hw_sgl_sge() is in the task process, obtain the value of le16_to_cpu(hw_sgl->entry_sum_in_sgl) before loop execting to shorten the loop execution time. Signed-off-by: Weili Qian <qianweili@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-05mm, treewide: redefine MAX_ORDER sanelyKirill A. Shutemov
MAX_ORDER currently defined as number of orders page allocator supports: user can ask buddy allocator for page order between 0 and MAX_ORDER-1. This definition is counter-intuitive and lead to number of bugs all over the kernel. Change the definition of MAX_ORDER to be inclusive: the range of orders user can ask from buddy allocator is 0..MAX_ORDER now. [kirill@shutemov.name: fix min() warning] Link: https://lkml.kernel.org/r/20230315153800.32wib3n5rickolvh@box [akpm@linux-foundation.org: fix another min_t warning] [kirill@shutemov.name: fixups per Zi Yan] Link: https://lkml.kernel.org/r/20230316232144.b7ic4cif4kjiabws@box.shutemov.name [akpm@linux-foundation.org: fix underlining in docs] Link: https://lore.kernel.org/oe-kbuild-all/202303191025.VRCTk6mP-lkp@intel.com/ Link: https://lkml.kernel.org/r/20230315113133.11326-11-kirill.shutemov@linux.intel.com Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Reviewed-by: Michael Ellerman <mpe@ellerman.id.au> [powerpc] Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-02-21Merge tag 'v6.3-p1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto update from Herbert Xu: "API: - Use kmap_local instead of kmap_atomic - Change request callback to take void pointer - Print FIPS status in /proc/crypto (when enabled) Algorithms: - Add rfc4106/gcm support on arm64 - Add ARIA AVX2/512 support on x86 Drivers: - Add TRNG driver for StarFive SoC - Delete ux500/hash driver (subsumed by stm32/hash) - Add zlib support in qat - Add RSA support in aspeed" * tag 'v6.3-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (156 commits) crypto: x86/aria-avx - Do not use avx2 instructions crypto: aspeed - Fix modular aspeed-acry crypto: hisilicon/qm - fix coding style issues crypto: hisilicon/qm - update comments to match function crypto: hisilicon/qm - change function names crypto: hisilicon/qm - use min() instead of min_t() crypto: hisilicon/qm - remove some unused defines crypto: proc - Print fips status crypto: crypto4xx - Call dma_unmap_page when done crypto: octeontx2 - Fix objects shared between several modules crypto: nx - Fix sparse warnings crypto: ecc - Silence sparse warning tls: Pass rec instead of aead_req into tls_encrypt_done crypto: api - Remove completion function scaffolding tls: Remove completion function scaffolding tipc: Remove completion function scaffolding net: ipv6: Remove completion function scaffolding net: ipv4: Remove completion function scaffolding net: macsec: Remove completion function scaffolding dm: Remove completion function scaffolding ...
2023-02-14crypto: hisilicon/qm - fix coding style issuesWeili Qian
1. Remove extra blank lines. 2. Remove extra spaces. 3. Use spaces instead of tabs around '=' and '\', to ensure consistent coding styles. 4. Macros should be capital letters, change 'QM_SQC_VFT_NUM_MASK_v2' to 'QM_SQC_VFT_NUM_MASK_V2'. Signed-off-by: Weili Qian <qianweili@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-01-27crypto: hisilicon: Wipe entire pool on errorKees Cook
To work around a Clang __builtin_object_size bug that shows up under CONFIG_FORTIFY_SOURCE and UBSAN_BOUNDS, move the per-loop-iteration mem_block wipe into a single wipe of the entire pool structure after the loop. Reported-by: Nathan Chancellor <nathan@kernel.org> Link: https://github.com/ClangBuiltLinux/linux/issues/1780 Cc: Weili Qian <qianweili@huawei.com> Cc: Zhou Wang <wangzhou1@hisilicon.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: linux-crypto@vger.kernel.org Signed-off-by: Kees Cook <keescook@chromium.org> Tested-by: Nathan Chancellor <nathan@kernel.org> # build Link: https://lore.kernel.org/r/20230106041945.never.831-kees@kernel.org
2022-04-15crypto: hisilicon/sgl - align the hardware sgl dma addressYang Shen
The hardware needs aligned sgl dma address. So expend the sgl_size to align 64 bytes. Signed-off-by: Yang Shen <shenyang39@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-03-15crypto: hisilicon/qm: Move the QM header to include/linuxShameer Kolothum
Since we are going to introduce VFIO PCI HiSilicon ACC driver for live migration in subsequent patches, move the ACC QM header file to a common include dir. Acked-by: Zhou Wang <wangzhou1@hisilicon.com> Acked-by: Longfang Liu <liulongfang@huawei.com> Acked-by: Kai Ye <yekai13@huawei.com> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Link: https://lore.kernel.org/r/20220308184902.2242-2-shameerali.kolothum.thodi@huawei.com Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2021-04-16crypto: hisilicon/sgl - fix the sg buf unmapKai Ye
Add data cleared operation for sge data. Signed-off-by: Kai Ye <yekai13@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-04-16crypto: hisilicon/sgl - fix the soft sg map to hardware sgKai Ye
The buffer of the hardware sge needs to be initialized by soft sgl. Signed-off-by: Kai Ye <yekai13@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-04-16crypto: hisilicon/sgl - add some dfx logsKai Ye
Add some dfx logs in some abnormal exit situations. Signed-off-by: Kai Ye <yekai13@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-04-16crypto: hisilicon/sgl - delete unneeded variable initializationKai Ye
Delete unneeded variable initialization Signed-off-by: Kai Ye <yekai13@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-04-16crypto: hisilicon/sgl - add a comment for block size initializationKai Ye
This seems "32" and "31" is obfuscating, It might be better to add a comment, which explain it. Signed-off-by: Kai Ye <yekai13@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-10-30crypto: hisilicon - Fix doc warnings in sgl.c and qm.cShiju Jose
Fix following warnings caused by mismatch between function parameters and function comments. drivers/crypto/hisilicon/sgl.c:256: warning: Excess function parameter 'hw_sgl_dma' description in 'hisi_acc_sg_buf_unmap' drivers/crypto/hisilicon/sgl.c:256: warning: Excess function parameter 'pool' description in 'hisi_acc_sg_buf_unmap' drivers/crypto/hisilicon/qm.c:1849: warning: Function parameter or member 'qp' not described in 'qm_drain_qp' drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'qm' not described in 'hisi_qm_set_vft' drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'fun_num' not described in 'hisi_qm_set_vft' drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'base' not described in 'hisi_qm_set_vft' drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'number' not described in 'hisi_qm_set_vft' drivers/crypto/hisilicon/qm.c:2620: warning: Function parameter or member 'qm' not described in 'qm_clear_queues' Signed-off-by: Shiju Jose <shiju.jose@huawei.com> Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-06-15crypto: hisilicon - Cap block size at 2^31Herbert Xu
The function hisi_acc_create_sg_pool may allocate a block of memory of size PAGE_SIZE * 2^(MAX_ORDER - 1). This value may exceed 2^31 on ia64, which would overflow the u32. This patch caps it at 2^31. Reported-by: kernel test robot <lkp@intel.com> Fixes: d8ac7b85236b ("crypto: hisilicon - fix large sgl memory...") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-12-11crypto: hisilicon - Remove useless MODULE macrosZhou Wang
As we already merge hardware sgl into hisi_qm module, remove useless MODULE macros. Fixes: 48c1cd40fae3 (crypto: hisilicon - merge sgl support to hisi_qm module) Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-12-11crypto: hisilicon - Fix issue with wrong number of sg elements after dma mapJonathan Cameron
We fill the hardware scatter gather list assuming it will need the same number of elements at the original scatterlist. If an IOMMU is involved, then it may well need fewer. The return value of dma_map_sg tells us how many. Probably never caused visible problems as the hardware won't get to the elements that are incorrect before it finds enough space. Fixes: dfed0098ab91 (crypto: hisilicon - add hardware SGL support) Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-11-01crypto: hisilicon - use sgl API to get sgl dma addr and lenZhou Wang
Use sgl API to get sgl dma addr and len, this will help to avoid compile error in some platforms. So NEED_SG_DMA_LENGTH can be removed here, which can only be selected by arch code. Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Suggested-by: Herbert Xu <herbert@gondor.apana.org.au> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-10-10crypto: hisilicon - misc fix about sglZhou Wang
This patch fixes some misc problems in sgl codes, e.g. missing static, sparse error and input parameter check. Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Shukun Tan <tanshukun1@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-10-10crypto: hisilicon - fix large sgl memory allocation problem when disable smmuZhou Wang
When disabling SMMU, it may fail to allocate large continuous memory. This patch fixes this by allocating memory as blocks. Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Shukun Tan <tanshukun1@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-10-10crypto: hisilicon - add sgl_sge_nr module param for zipShukun Tan
Add a module parameter for zip driver to set the number of SGE in one SGL. Signed-off-by: Shukun Tan <tanshukun1@huawei.com> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-10-10crypto: hisilicon - merge sgl support to hisi_qm moduleZhou Wang
As HW SGL can be seen as a data format of QM's sqe, we merge sgl code into qm module and rename it as hisi_qm, which reduces the number of module and make the name less generic. This patch also modify the interface of SGL: - Create/free hisi_acc_sgl_pool inside. - Let user to pass the SGE number in one SGL when creating sgl pool, which is better than a unified module parameter for sgl module before. - Modify zip driver according to sgl interface change. Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Shukun Tan <tanshukun1@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-15crypto: hisilicon - init curr_sgl_dma to fix compile warningZhou Wang
Just init curr_sgl_dma = 0 to avoid compile warning. Fixes: dfed0098ab91 ("crypto: hisilicon - add hardware SGL support") Reported-by: kbuild test robot <lkp@intel.com> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-08-09crypto: hisilicon - add hardware SGL supportZhou Wang
HiSilicon accelerators in Hip08 use same hardware scatterlist for data format. We support it in this module. Specific accelerator drivers can use hisi_acc_create_sgl_pool to allocate hardware SGLs ahead. Then use hisi_acc_sg_buf_map_to_hw_sgl to get one hardware SGL and pass related information to hardware SGL. The DMA address of mapped hardware SGL can be passed to SGL src/dst field in QM SQE. Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
generated by cgit (git 2.34.1) at 2025-08-04 09:50:51 +0000