summaryrefslogtreecommitdiff
path: root/drivers/crypto/amcc
AgeCommit message (Collapse)Author
2019-05-03crypto: crypto4xx - get rid of redundant using_sd variableChristian Lamparter
using_sd is used as a stand-in for sa_command_0.bf.scatter that we need to set anyway, so we might as well just prevent double-accounting. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-03crypto: crypto4xx - use sync skcipher for fallbackChristian Lamparter
This replaces struct crypto_skcipher and the extra request size with struct crypto_sync_skcipher and SYNC_SKCIPHER_REQUEST_ON_STACK(), which uses a fixed stack size. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-03crypto: crypto4xx - fix cfb and ofb "overran dst buffer" issuesChristian Lamparter
Currently, crypto4xx CFB and OFB AES ciphers are failing testmgr's test vectors. |cfb-aes-ppc4xx encryption overran dst buffer on test vector 3, cfg="in-place" |ofb-aes-ppc4xx encryption overran dst buffer on test vector 1, cfg="in-place" This is because of a very subtile "bug" in the hardware that gets indirectly mentioned in 18.1.3.5 Encryption/Decryption of the hardware spec: the OFB and CFB modes for AES are listed there as operation modes for >>> "Block ciphers" <<<. Which kind of makes sense, but we would like them to be considered as stream ciphers just like the CTR mode. To workaround this issue and stop the hardware from causing "overran dst buffer" on crypttexts that are not a multiple of 16 (AES_BLOCK_SIZE), we force the driver to use the scatter buffers as the go-between. As a bonus this patch also kills redundant pd_uinfo->num_gd and pd_uinfo->num_sd setters since the value has already been set before. Cc: stable@vger.kernel.org Fixes: f2a13e7cba9e ("crypto: crypto4xx - enable AES RFC3686, ECB, CFB and OFB offloads") Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-05-03crypto: crypto4xx - fix ctr-aes missing output IVChristian Lamparter
Commit 8efd972ef96a ("crypto: testmgr - support checking skcipher output IV") caused the crypto4xx driver to produce the following error: | ctr-aes-ppc4xx encryption test failed (wrong output IV) | on test vector 0, cfg="in-place" This patch fixes this by reworking the crypto4xx_setkey_aes() function to: - not save the iv for ECB (as per 18.2.38 CRYP0_SA_CMD_0: "This bit mut be cleared for DES ECB mode or AES ECB mode, when no IV is used.") - instruct the hardware to save the generated IV for all other modes of operations that have IV and then supply it back to the callee in pretty much the same way as we do it for cbc-aes already. - make it clear that the DIR_(IN|OUT)BOUND is the important bit that tells the hardware to encrypt or decrypt the data. (this is cosmetic - but it hopefully prevents me from getting confused again). - don't load any bogus hash when we don't use any hash operation to begin with. Cc: stable@vger.kernel.org Fixes: f2a13e7cba9e ("crypto: crypto4xx - enable AES RFC3686, ECB, CFB and OFB offloads") Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-03-05Merge branch 'linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto update from Herbert Xu: "API: - Add helper for simple skcipher modes. - Add helper to register multiple templates. - Set CRYPTO_TFM_NEED_KEY when setkey fails. - Require neither or both of export/import in shash. - AEAD decryption test vectors are now generated from encryption ones. - New option CONFIG_CRYPTO_MANAGER_EXTRA_TESTS that includes random fuzzing. Algorithms: - Conversions to skcipher and helper for many templates. - Add more test vectors for nhpoly1305 and adiantum. Drivers: - Add crypto4xx prng support. - Add xcbc/cmac/ecb support in caam. - Add AES support for Exynos5433 in s5p. - Remove sha384/sha512 from artpec7 as hardware cannot do partial hash" [ There is a merge of the Freescale SoC tree in order to pull in changes required by patches to the caam/qi2 driver. ] * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (174 commits) crypto: s5p - add AES support for Exynos5433 dt-bindings: crypto: document Exynos5433 SlimSSS crypto: crypto4xx - add missing of_node_put after of_device_is_available crypto: cavium/zip - fix collision with generic cra_driver_name crypto: af_alg - use struct_size() in sock_kfree_s() crypto: caam - remove redundant likely/unlikely annotation crypto: s5p - update iv after AES-CBC op end crypto: x86/poly1305 - Clear key material from stack in SSE2 variant crypto: caam - generate hash keys in-place crypto: caam - fix DMA mapping xcbc key twice crypto: caam - fix hash context DMA unmap size hwrng: bcm2835 - fix probe as platform device crypto: s5p-sss - Use AES_BLOCK_SIZE define instead of number crypto: stm32 - drop pointless static qualifier in stm32_hash_remove() crypto: chelsio - Fixed Traffic Stall crypto: marvell - Remove set but not used variable 'ivsize' crypto: ccp - Update driver messages to remove some confusion crypto: adiantum - add 1536 and 4096-byte test vectors crypto: nhpoly1305 - add a test vector with len % 16 != 0 crypto: arm/aes-ce - update IV after partial final CTR block ...
2019-02-28crypto: crypto4xx - add missing of_node_put after of_device_is_availableJulia Lawall
Add an of_node_put when a tested device node is not available. The semantic patch that fixes this problem is as follows (http://coccinelle.lip6.fr): // <smpl> @@ identifier f; local idexpression e; expression x; @@ e = f(...); ... when != of_node_put(e) when != x = e when != e = x when any if (<+...of_device_is_available(e)...+>) { ... when != of_node_put(e) ( return e; | + of_node_put(e); return ...; ) } // </smpl> Fixes: 5343e674f32fb ("crypto4xx: integrate ppc4xx-rng into crypto4xx") Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-02-01crypto: crypto4xx - Fix wrong ppc4xx_trng_probe()/ppc4xx_trng_remove() argumentsCorentin Labbe
When building without CONFIG_HW_RANDOM_PPC4XX, I hit the following build failure: drivers/crypto/amcc/crypto4xx_core.c: In function 'crypto4xx_probe': drivers/crypto/amcc/crypto4xx_core.c:1407:20: error: passing argument 1 of 'ppc4xx_trng_probe' from incompatible pointer type [-Werror=incompatible-pointer-types] In file included from drivers/crypto/amcc/crypto4xx_core.c:50:0: drivers/crypto/amcc/crypto4xx_trng.h:28:20: note: expected 'struct crypto4xx_device *' but argument is of type 'struct crypto4xx_core_device *' drivers/crypto/amcc/crypto4xx_core.c: In function 'crypto4xx_remove': drivers/crypto/amcc/crypto4xx_core.c:1434:21: error: passing argument 1 of 'ppc4xx_trng_remove' from incompatible pointer type [-Werror=incompatible-pointer-types] In file included from drivers/crypto/amcc/crypto4xx_core.c:50:0: drivers/crypto/amcc/crypto4xx_trng.h:30:20: note: expected 'struct crypto4xx_device *' but argument is of type 'struct crypto4xx_core_device *' This patch fix the needed argument of ppc4xx_trng_probe()/ppc4xx_trng_remove() in that case. Fixes: 5343e674f32f ("crypto4xx: integrate ppc4xx-rng into crypto4xx") Signed-off-by: Corentin Labbe <clabbe@baylibre.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-01-11crypto: crypto4xx - add prng crypto supportChristian Lamparter
This patch adds support for crypto4xx's ANSI X9.17 Annex C compliant pseudo random number generator which provides a pseudo random source for the purpose of generating Initialization Vectors (IV's) for AES algorithms to the Packet Engine and other pseudo random number requirements. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2019-01-08cross-tree: phase out dma_zalloc_coherent()Luis Chamberlain
We already need to zero out memory for dma_alloc_coherent(), as such using dma_zalloc_coherent() is superflous. Phase it out. This change was generated with the following Coccinelle SmPL patch: @ replace_dma_zalloc_coherent @ expression dev, size, data, handle, flags; @@ -dma_zalloc_coherent(dev, size, handle, flags) +dma_alloc_coherent(dev, size, handle, flags) Suggested-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> [hch: re-ran the script on the latest tree] Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-27Merge branch 'linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto updates from Herbert Xu: "API: - Add 1472-byte test to tcrypt for IPsec - Reintroduced crypto stats interface with numerous changes - Support incremental algorithm dumps Algorithms: - Add xchacha12/20 - Add nhpoly1305 - Add adiantum - Add streebog hash - Mark cts(cbc(aes)) as FIPS allowed Drivers: - Improve performance of arm64/chacha20 - Improve performance of x86/chacha20 - Add NEON-accelerated nhpoly1305 - Add SSE2 accelerated nhpoly1305 - Add AVX2 accelerated nhpoly1305 - Add support for 192/256-bit keys in gcmaes AVX - Add SG support in gcmaes AVX - ESN for inline IPsec tx in chcr - Add support for CryptoCell 703 in ccree - Add support for CryptoCell 713 in ccree - Add SM4 support in ccree - Add SM3 support in ccree - Add support for chacha20 in caam/qi2 - Add support for chacha20 + poly1305 in caam/jr - Add support for chacha20 + poly1305 in caam/qi2 - Add AEAD cipher support in cavium/nitrox" * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (130 commits) crypto: skcipher - remove remnants of internal IV generators crypto: cavium/nitrox - Fix build with !CONFIG_DEBUG_FS crypto: salsa20-generic - don't unnecessarily use atomic walk crypto: skcipher - add might_sleep() to skcipher_walk_virt() crypto: x86/chacha - avoid sleeping under kernel_fpu_begin() crypto: cavium/nitrox - Added AEAD cipher support crypto: mxc-scc - fix build warnings on ARM64 crypto: api - document missing stats member crypto: user - remove unused dump functions crypto: chelsio - Fix wrong error counter increments crypto: chelsio - Reset counters on cxgb4 Detach crypto: chelsio - Handle PCI shutdown event crypto: chelsio - cleanup:send addr as value in function argument crypto: chelsio - Use same value for both channel in single WR crypto: chelsio - Swap location of AAD and IV sent in WR crypto: chelsio - remove set but not used variable 'kctx_len' crypto: ux500 - Use proper enum in hash_set_dma_transfer crypto: ux500 - Use proper enum in cryp_set_dma_transfer crypto: aesni - Add scatter/gather avx stubs, and use them in C crypto: aesni - Introduce partial block macro ..
2018-12-20crypto4xx_core: don't abuse __dma_sync_pageChristoph Hellwig
This function is internal to the DMA API implementation. Instead use the DMA API to properly unmap. Note that the DMA API usage in this driver is a disaster and urgently needs some work - it is missing all the unmaps, seems to do a secondary map where it looks like it should to a unmap in one place to work around cache coherency and the directions passed in seem to be partially wrong. Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-11-20crypto: drop mask=CRYPTO_ALG_ASYNC from 'cipher' tfm allocationsEric Biggers
'cipher' algorithms (single block ciphers) are always synchronous, so passing CRYPTO_ALG_ASYNC in the mask to crypto_alloc_cipher() has no effect. Many users therefore already don't pass it, but some still do. This inconsistency can cause confusion, especially since the way the 'mask' argument works is somewhat counterintuitive. Thus, just remove the unneeded CRYPTO_ALG_ASYNC flags. This patch shouldn't change any actual behavior. Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-07-09crypto: skcipher - remove useless setting of type flagsEric Biggers
Some skcipher algorithms set .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER. But this is redundant with the C structure type ('struct skcipher_alg'), and crypto_register_skcipher() already sets the type flag automatically, clearing any type flag that was already there. Apparently the useless assignment has just been copy+pasted around. So, remove the useless assignment from all the skcipher algorithms. This patch shouldn't change any actual behavior. Signed-off-by: Eric Biggers <ebiggers@google.com> Acked-by: Gilad Ben-Yossef <gilad@benyossef.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-06-12treewide: kzalloc() -> kcalloc()Kees Cook
The kzalloc() function has a 2-factor argument form, kcalloc(). This patch replaces cases of: kzalloc(a * b, gfp) with: kcalloc(a * b, gfp) as well as handling cases of: kzalloc(a * b * c, gfp) with: kzalloc(array3_size(a, b, c), gfp) as it's slightly less ugly than: kzalloc_array(array_size(a, b), c, gfp) This does, however, attempt to ignore constant size factors like: kzalloc(4 * 1024, gfp) though any constants defined via macros get caught up in the conversion. Any factors with a sizeof() of "unsigned char", "char", and "u8" were dropped, since they're redundant. The Coccinelle script used for this was: // Fix redundant parens around sizeof(). @@ type TYPE; expression THING, E; @@ ( kzalloc( - (sizeof(TYPE)) * E + sizeof(TYPE) * E , ...) | kzalloc( - (sizeof(THING)) * E + sizeof(THING) * E , ...) ) // Drop single-byte sizes and redundant parens. @@ expression COUNT; typedef u8; typedef __u8; @@ ( kzalloc( - sizeof(u8) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(__u8) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(char) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(unsigned char) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(u8) * COUNT + COUNT , ...) | kzalloc( - sizeof(__u8) * COUNT + COUNT , ...) | kzalloc( - sizeof(char) * COUNT + COUNT , ...) | kzalloc( - sizeof(unsigned char) * COUNT + COUNT , ...) ) // 2-factor product with sizeof(type/expression) and identifier or constant. @@ type TYPE; expression THING; identifier COUNT_ID; constant COUNT_CONST; @@ ( - kzalloc + kcalloc ( - sizeof(TYPE) * (COUNT_ID) + COUNT_ID, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * COUNT_ID + COUNT_ID, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * (COUNT_CONST) + COUNT_CONST, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * COUNT_CONST + COUNT_CONST, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (COUNT_ID) + COUNT_ID, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * COUNT_ID + COUNT_ID, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (COUNT_CONST) + COUNT_CONST, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * COUNT_CONST + COUNT_CONST, sizeof(THING) , ...) ) // 2-factor product, only identifiers. @@ identifier SIZE, COUNT; @@ - kzalloc + kcalloc ( - SIZE * COUNT + COUNT, SIZE , ...) // 3-factor product with 1 sizeof(type) or sizeof(expression), with // redundant parens removed. @@ expression THING; identifier STRIDE, COUNT; type TYPE; @@ ( kzalloc( - sizeof(TYPE) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(THING) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) ) // 3-factor product with 2 sizeof(variable), with redundant parens removed. @@ expression THING1, THING2; identifier COUNT; type TYPE1, TYPE2; @@ ( kzalloc( - sizeof(TYPE1) * sizeof(TYPE2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kzalloc( - sizeof(THING1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kzalloc( - sizeof(THING1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) ) // 3-factor product, only identifiers, with redundant parens removed. @@ identifier STRIDE, SIZE, COUNT; @@ ( kzalloc( - (COUNT) * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) ) // Any remaining multi-factor products, first at least 3-factor products, // when they're not all constants... @@ expression E1, E2, E3; constant C1, C2, C3; @@ ( kzalloc(C1 * C2 * C3, ...) | kzalloc( - (E1) * E2 * E3 + array3_size(E1, E2, E3) , ...) | kzalloc( - (E1) * (E2) * E3 + array3_size(E1, E2, E3) , ...) | kzalloc( - (E1) * (E2) * (E3) + array3_size(E1, E2, E3) , ...) | kzalloc( - E1 * E2 * E3 + array3_size(E1, E2, E3) , ...) ) // And then all remaining 2 factors products when they're not all constants, // keeping sizeof() as the second factor argument. @@ expression THING, E1, E2; type TYPE; constant C1, C2, C3; @@ ( kzalloc(sizeof(THING) * C2, ...) | kzalloc(sizeof(TYPE) * C2, ...) | kzalloc(C1 * C2 * C3, ...) | kzalloc(C1 * C2, ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * (E2) + E2, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * E2 + E2, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (E2) + E2, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * E2 + E2, sizeof(THING) , ...) | - kzalloc + kcalloc ( - (E1) * E2 + E1, E2 , ...) | - kzalloc + kcalloc ( - (E1) * (E2) + E1, E2 , ...) | - kzalloc + kcalloc ( - E1 * E2 + E1, E2 , ...) ) Signed-off-by: Kees Cook <keescook@chromium.org>
2018-04-28crypto: crypto4xx - put temporary dst sg into request ctxChristian Lamparter
This patch fixes a crash that happens when testing rfc4543(gcm(aes)) Unable to handle kernel paging request for data at address 0xf59b3420 Faulting instruction address: 0xc0012994 Oops: Kernel access of bad area, sig: 11 [#1] BE PowerPC 44x Platform Modules linked in: tcrypt(+) crypto4xx [...] CPU: 0 PID: 0 Comm: swapper Tainted: G O 4.17.0-rc1+ #23 NIP: c0012994 LR: d3077934 CTR: 06026d49 REGS: cfff7e30 TRAP: 0300 Tainted: G O (4.17.0-rc1+) MSR: 00029000 <CE,EE,ME> CR: 44744822 XER: 00000000 DEAR: f59b3420 ESR: 00000000 NIP [c0012994] __dma_sync+0x58/0x10c LR [d3077934] crypto4xx_bh_tasklet_cb+0x188/0x3c8 [crypto4xx] __dma_sync was fed the temporary _dst that crypto4xx_build_pd() had in it's function stack. This clearly never worked. This patch therefore overhauls the code from the original driver and puts the temporary dst sg list into aead's request context. Fixes: a0aae821ba3d3 ("crypto: crypto4xx - prepare for AEAD support") Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-04-28crypto: crypto4xx - extend aead fallback checksChristian Lamparter
1020 bytes is the limit for associated data. Any more and it will no longer fit into hash_crypto_offset anymore. The hardware will not process aead requests with plaintext that have less than AES_BLOCK_SIZE bytes. When decrypting aead requests the authsize has to be taken in account as well, as it is part of the cryptlen. Otherwise the hardware will think it has been misconfigured and will return: aead return err status = 0x98 For rtc4543(gcm(aes)), the hardware has a dedicated GMAC mode as part of the hash function set. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-04-28crypto: crypto4xx - properly set IV after de- and encryptChristian Lamparter
This patch fixes cts(cbc(aes)) test when cbc-aes-ppc4xx is used. alg: skcipher: Test 1 failed (invalid result) on encryption for cts(cbc-aes-ppc4xx) 00000000: 4b 10 75 fc 2f 14 1b 6a 27 35 37 33 d1 b7 70 05 00000010: 97 alg: skcipher: Failed to load transform for cts(cbc(aes)): -2 The CTS cipher mode expect the IV (req->iv) of skcipher_request to contain the last ciphertext block after the {en,de}crypt operation is complete. Fix this issue for the AMCC Crypto4xx hardware engine. The tcrypt test case for cts(cbc(aes)) is now correctly passed. name : cts(cbc(aes)) driver : cts(cbc-aes-ppc4xx) module : cts priority : 300 refcnt : 1 selftest : passed internal : no type : skcipher async : yes blocksize : 16 min keysize : 16 max keysize : 32 ivsize : 16 chunksize : 16 walksize : 16 Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-04-28crypto: crypto4xx - add aes-ctr supportChristian Lamparter
This patch adds support for the aes-ctr skcipher. name : ctr(aes) driver : ctr-aes-ppc4xx module : crypto4xx priority : 300 refcnt : 1 selftest : passed internal : no type : skcipher async : yes blocksize : 16 min keysize : 16 max keysize : 32 ivsize : 16 chunksize : 16 walksize : 16 The hardware uses only the last 32-bits as the counter while the kernel tests (aes_ctr_enc_tv_template[4] for example) expect that the whole IV is a counter. To make this work, the driver will fallback if the counter is going to overlow. The aead's crypto4xx_setup_fallback() function is renamed to crypto4xx_aead_setup_fallback. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-04-28crypto: crypto4xx - avoid VLA useChristian Lamparter
This patch fixes some of the -Wvla warnings. crypto4xx_alg.c:83:19: warning: Variable length array is used. crypto4xx_alg.c:273:56: warning: Variable length array is used. crypto4xx_alg.c:380:32: warning: Variable length array is used. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-04-28crypto: crypto4xx - convert to skcipherChristian Lamparter
The ablkcipher APIs have been effectively deprecated since [1]. This patch converts the crypto4xx driver to the new skcipher APIs. [1] <https://www.spinics.net/lists/linux-crypto/msg18133.html> Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-04-28crypto: crypto4xx - performance optimizationsChristian Lamparter
This patch provides a cheap 2MiB/s+ (~ 6%) performance improvement over the current code. This is because the compiler can now optimize several endian swap memcpy. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-12crypto: Use zeroing memory allocator instead of allocator/memsetHimanshu Jha
Use dma_zalloc_coherent for allocating zeroed memory and remove unnecessary memset function. Done using Coccinelle. Generated-by: scripts/coccinelle/api/alloc/kzalloc-simple.cocci 0-day tested with no failures. Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-05crypto: crypto4xx - perform aead icv check in the driverChristian Lamparter
The ccm-aes-ppc4xx now fails one of testmgr's expected failure test cases as such: |decryption failed on test 10 for ccm-aes-ppc4xx: |ret was 0, |expected -EBADMSG It doesn't look like the hardware sets the authentication failure flag. The original vendor source from which this was ported does not have any special code or notes about why this would happen or if there are any WAs. Hence, this patch converts the aead_done callback handler to perform the icv check in the driver. And this fixes the false negative and the ccm-aes-ppc4xx passes the selftests once again. |name : ccm(aes) |driver : ccm-aes-ppc4xx |module : crypto4xx |priority : 300 |refcnt : 1 |selftest : passed |internal : no |type : aead |async : yes |blocksize : 1 |ivsize : 16 |maxauthsize : 16 |geniv : <none> Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-05crypto: crypto4xx - kill MODULE_NAMEChristian Lamparter
KBUILD_MODNAME provides the same value. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-05crypto: crypto4xx - fix missing irq devnameChristian Lamparter
crypto4xx_device's name variable is not set to anything. The common devname for request_irq seems to be the module name. This will fix the seemingly anonymous interrupt entry in /proc/interrupts for crypto4xx. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-05crypto: crypto4xx - support Revision B partsChristian Lamparter
This patch adds support for the crypto4xx RevB cores found in the 460EX, 460SX and later cores (like the APM821xx). Without this patch, the crypto4xx driver will not be able to process any offloaded requests and simply hang indefinitely. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2018-01-05crypto: crypto4xx - shuffle iomap in front of request_irqChristian Lamparter
It is possible to avoid the ce_base null pointer check in the drivers' interrupt handler routine "crypto4xx_ce_interrupt_handler()" by simply doing the iomap in front of the IRQ registration. This way, the ce_base will always be valid in the handler and a branch in an critical path can be avoided. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - add aes-gcm supportChristian Lamparter
This patch adds aes-gcm support to crypto4xx. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - add aes-ccm supportChristian Lamparter
This patch adds aes-ccm support. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - prepare for AEAD supportChristian Lamparter
This patch enhances existing interfaces and functions to support AEAD ciphers in the next patches. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - simplify sa and state context acquisitionChristian Lamparter
Thanks to the big overhaul of crypto4xx_build_pd(), the request-local sa_in, sa_out and state_record allocation can be simplified. There's no need to setup any dma coherent memory anymore and much of the support code can be removed. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - fix stalls under heavy loadChristian Lamparter
If the crypto4xx device is continuously loaded by dm-crypt and ipsec work, it will start to work intermittent after a few (between 20-30) seconds, hurting throughput and latency. This patch contains various stability improvements in order to fix this issue. So far, the hardware has survived more than a day without suffering any stalls under the continuous load. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - fix various warningsChristian Lamparter
crypto4xx_core.c:179:6: warning: symbol 'crypto4xx_free_state_record' was not declared. Should it be static? crypto4xx_core.c:331:5: warning: symbol 'crypto4xx_get_n_gd' was not declared. Should it be static? crypto4xx_core.c:652:6: warning: symbol 'crypto4xx_return_pd' was not declared. Should it be static? crypto4xx_return_pd() is not used by anything. Therefore it is removed. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - overhaul crypto4xx_build_pd()Christian Lamparter
This patch overhauls and fixes code related to crypto4xx_build_pd() * crypto4xx_build_pd() did not handle chained source scatterlist. This is fixed by replacing the buggy indexed-access of &src[idx] with sg_next() in the gather array setup loop. * The redundant is_hash, direction, save_iv and pd_ctl members in the crypto4xx_ctx struct have been removed. - is_hash can be derived from the crypto_async_request parameter. - direction is already part of the security association's bf.dir bitfield. - save_iv is unused. - pd_ctl always had the host_ready bit enabled anyway. (the hash_final case is rather pointless, since the ahash code has been deactivated). * make crypto4xx_build_pd()'s caller responsible for converting the IV to the LE32 format. * change crypto4xx_ahash_update() and crypto4xx_ahash_digest() to initialize a temporary destination scatterlist. This allows the removal of an ugly cast of req->result (which is a pointer to an u8-array) to a scatterlist pointer. * change crypto4xx_build_pd() return type to int. After all it returns -EINPROGRESS/-EBUSY. * fix crypto4xx_build_pd() thread-unsafe sa handling. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - use the correct LE32 format for IV and key defsChristian Lamparter
The hardware expects that the keys, IVs (and inner/outer hashes) are in the le32 format. This patch changes all hardware interface declarations to use the correct LE32 data format for each field. In order to pass __CHECK_ENDIAN__ checks, crypto4xx_memcpy_le has to be honest about the endianness of its parameters. The function was split and moved to the common crypto4xx_core.h header. This allows the compiler to generate better code if the sizes/len is a constant (various *_IV_LEN). Please note that the hardware isn't consistent with the endiannes of the save_digest field in the state record struct though. The hashes produced by GHASH and CBC (for CCM) will be in LE32. Whereas md5 and sha{1/,256,...} do not need any conversion. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - add backlog queue supportChristian Lamparter
Previously, If the crypto4xx driver used all available security contexts, it would simply refuse new requests with -EAGAIN. CRYPTO_TFM_REQ_MAY_BACKLOG was ignored. in case of dm-crypt.c's crypt_convert() function this was causing the following errors to manifest, if the system was pushed hard enough: | EXT4-fs warning (dm-1): ext4_end_bio:314: I/O error -5 writing to ino .. | EXT4-fs warning (dm-1): ext4_end_bio:314: I/O error -5 writing to ino .. | EXT4-fs warning (dm-1): ext4_end_bio:314: I/O error -5 writing to ino .. | JBD2: Detected IO errors while flushing file data on dm-1-8 | Aborting journal on device dm-1-8. | EXT4-fs error : ext4_journal_check_start:56: Detected aborted journal | EXT4-fs (dm-1): Remounting filesystem read-only | EXT4-fs : ext4_writepages: jbd2_start: 2048 pages, inode 498...; err -30 (This did cause corruptions due to failed writes) To fix this mess, the crypto4xx driver needs to notifiy the user to slow down. This can be achieved by returning -EBUSY on requests, once the crypto hardware was falling behind. Note: -EBUSY has two different meanings. Setting the flag CRYPTO_TFM_REQ_MAY_BACKLOG implies that the request was successfully queued, by the crypto driver. To achieve this requirement, the implementation introduces a threshold check and adds logic to the completion routines in much the same way as AMD's Cryptographic Coprocessor (CCP) driver do. Note2: Tests showed that dm-crypt starved ipsec traffic. Under load, ipsec links dropped to 0 Kbits/s. This is because dm-crypt's callback would instantly queue the next request. In order to not starve ipsec, the driver reserves a small portion of the available crypto contexts for this purpose. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - increase context and scatter ring buffer elementsChristian Lamparter
If crypto4xx is used in conjunction with dm-crypt, the available ring buffer elements are not enough to handle the load properly. On an aes-cbc-essiv:sha256 encrypted swap partition the read performance is abyssal: (tested with hdparm -t) /dev/mapper/swap_crypt: Timing buffered disk reads: 14 MB in 3.68 seconds = 3.81 MB/sec The patch increases both PPC4XX_NUM_SD and PPC4XX_NUM_PD to 256. This improves the performance considerably: /dev/mapper/swap_crypt: Timing buffered disk reads: 104 MB in 3.03 seconds = 34.31 MB/sec Furthermore, PPC4XX_LAST_SD, PPC4XX_LAST_GD and PPC4XX_LAST_PD can be easily calculated from their respective PPC4XX_NUM_* constant. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - fix type mismatch compiler errorChristian Lamparter
This patch fixes a type mismatch error that I accidentally introduced when I moved and refactored the dynamic_contents helpers. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - fix off-by-one AES-OFBChristian Lamparter
I used aes-cbc as a template for ofb. But sadly I forgot to update set_key method to crypto4xx_setkey_aes_ofb(). this was caught by the testmgr: alg: skcipher: Test 1 failed (invalid result) on encr. for ofb-aes-ppc4xx 00000000: 76 49 ab ac 81 19 b2 46 ce e9 8e 9b 12 e9 19 7d 00000010: 50 86 cb 9b 50 72 19 ee 95 db 11 3a 91 76 78 b2 00000020: 73 be d6 b8 e3 c1 74 3b 71 16 e6 9e 22 22 95 16 00000030: 3f f1 ca a1 68 1f ac 09 12 0e ca 30 75 86 e1 a7 With the correct set_key method, the aes-ofb cipher passes the test. name : ofb(aes) driver : ofb-aes-ppc4xx module : crypto4xx priority : 300 refcnt : 1 selftest : passed internal : no type : ablkcipher async : yes blocksize : 16 min keysize : 16 max keysize : 32 ivsize : 16 geniv : <default> Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-10-12crypto: crypto4xx - wire up hmac_mc to hmac_mutingChristian Lamparter
The hmac_mc parameter of set_dynamic_sa_command_1() was defined but not used. On closer inspection it turns out, it was never wired up. Signed-off-by: Christian Lamparter <chunkeey@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - pointer arithmetic overhaulChristian Lamparter
This patch improves the readability of various functions, by replacing various void* pointers declarations with their respective structs *. This makes it possible to go for the eye-friendly array-indexing methods. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - fix crypto4xx_build_pdr, crypto4xx_build_sdr leakChristian Lamparter
If one of the later memory allocations in rypto4xx_build_pdr() fails: dev->pdr (and/or) dev->pdr_uinfo wouldn't be freed. crypto4xx_build_sdr() has the same issue with dev->sdr. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - replace crypto4xx_dev's scatter_buffer_size with constantChristian Lamparter
scatter_buffer_size is always set to PPC4XX_SD_BUFFER_SIZE. I don't think there's any point in keeping the variable around. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - refactor crypto4xx_copy_pkt_to_dst()Christian Lamparter
This patch refactors the crypto4xx_copy_pkt_to_dst() to use scatterwalk_map_and_copy() to copy the processed data between the crypto engine's scatter ring buffer and the destination specified by the ablkcipher_request. This also makes the crypto4xx_fill_one_page() function redundant. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - enable AES RFC3686, ECB, CFB and OFB offloadsChristian Lamparter
The crypto engine supports more than just aes-cbc. This patch enables the remaining AES block cipher modes that pass the testmanager's test vectors. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - move and refactor dynamic_contents helpersChristian Lamparter
This patch refactors and moves the dynamic_contents helper functions into the crypto4xx_sa.h header file. * get_dynamic_sa_iv_size is no longer needed, as the cryptoapi provides the required IV size information as well. * refactor the function declarations to use the a pointer to the dynamic_sa_contents union, instead of the crypto4xx_ctx. * rename get_dynamic_sa_offset_key_field to get_dynamic_sa_key_field. It returns the pointer to the key directly. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - fix dynamic_sa_ctl's sa_contents declarationChristian Lamparter
The driver had a union dynamic_sa_contents in place that described the meaning of the bits in the sa_contents variable. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - remove double assignment of pd_uinfo->stateChristian Lamparter
crypto4xx_put_pd_to_pdr() already clears the flag. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - remove extern statement before function declarationChristian Lamparter
All function declarations are "extern" by default, there is no need to specify it explicitly. For C99 states in 6.2.2.5: "If the declaration of an identifier for a function has no storage-class specifier, its linkage is determined exactly as if it were declared with the storage-class specifier extern." Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-09-22crypto: crypto4xx - set CRYPTO_ALG_KERN_DRIVER_ONLY flagChristian Lamparter
The security offload function is performed by a cryptographic engine core attached to the 128-bit PLB (processor local bus) with builtin DMA and interrupt controllers. This, I think, satisfies the requirement for the CRYPTO_ALG_KERN_DRIVER_ONLY flag. Signed-off-by: Christian Lamparter <chunkeey@googlemail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
generated by cgit (git 2.34.1) at 2024-06-24 07:59:08 +0000