summaryrefslogtreecommitdiff
path: root/arch
AgeCommit message (Collapse)Author
2015-04-03crypto: arm/sha256 - Add optimized SHA-256/224Sami Tolvanen
Add Andy Polyakov's optimized assembly and NEON implementations for SHA-256/224. The sha256-armv4.pl script for generating the assembly code is from OpenSSL commit 51f8d095562f36cdaa6893597b5c609e943b0565. Compared to sha256-generic these implementations have the following tcrypt speed improvements on Motorola Nexus 6 (Snapdragon 805): bs b/u sha256-neon sha256-asm 16 16 x1.32 x1.19 64 16 x1.27 x1.15 64 64 x1.36 x1.20 256 16 x1.22 x1.11 256 64 x1.36 x1.19 256 256 x1.59 x1.23 1024 16 x1.21 x1.10 1024 256 x1.65 x1.23 1024 1024 x1.76 x1.25 2048 16 x1.21 x1.10 2048 256 x1.66 x1.23 2048 1024 x1.78 x1.25 2048 2048 x1.79 x1.25 4096 16 x1.20 x1.09 4096 256 x1.66 x1.23 4096 1024 x1.79 x1.26 4096 4096 x1.82 x1.26 8192 16 x1.20 x1.09 8192 256 x1.67 x1.23 8192 1024 x1.80 x1.26 8192 4096 x1.85 x1.28 8192 8192 x1.85 x1.27 Where bs refers to block size and b/u to bytes per update. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Cc: Andy Polyakov <appro@openssl.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: sha-mb - mark Multi buffer SHA1 helper cipherStephan Mueller
Flag all Multi buffer SHA1 helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: arm64/aes - mark 64 bit ARMv8 AES helper ciphersStephan Mueller
Flag all 64 bit ARMv8 AES helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: aes-ce - mark ARMv8 AES helper ciphersStephan Mueller
Flag all ARMv8 AES helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: aesbs - mark NEON bit sliced AES helper ciphersStephan Mueller
Flag all NEON bit sliced AES helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: twofish_avx - mark Twofish AVX helper ciphersStephan Mueller
Flag all Twofish AVX helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: serpent_sse2 - mark Serpent SSE2 helper ciphersStephan Mueller
Flag all Serpent SSE2 helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: serpent_avx - mark Serpent AVX helper ciphersStephan Mueller
Flag all Serpent AVX helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: serpent_avx2 - mark Serpent AVX2 helper ciphersStephan Mueller
Flag all Serpent AVX2 helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: cast6_avx - mark CAST6 helper ciphersStephan Mueller
Flag all CAST6 helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: camellia_aesni_avx - mark AVX Camellia helper ciphersStephan Mueller
Flag all AVX Camellia helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: cast5_avx - mark CAST5 helper ciphersStephan Mueller
Flag all CAST5 helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: camellia_aesni_avx2 - mark AES-NI Camellia helper ciphersStephan Mueller
Flag all AES-NI Camellia helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: ghash-ce - mark GHASH ARMv8 vmull.p64 helper ciphersStephan Mueller
Flag all GHASH ARMv8 vmull.p64 helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: clmulni - mark ghash clmulni helper ciphersStephan Mueller
Flag all ash clmulni helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-31crypto: aesni - mark AES-NI helper ciphersStephan Mueller
Flag all AES-NI helper ciphers as internal ciphers to prevent them from being called by normal users. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-24crypto: arm/ghash - fix big-endian bug in ghashArd Biesheuvel
This fixes a bug in the new v8 Crypto Extensions GHASH code that only manifests itself in big-endian mode. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-16crypto: sha1-mb - Syntax errorAmeen Ali
fixing a syntax-error . Signed-off-by: Ameen Ali <AmeenAli023@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-13crypto: don't export static symbolJulia Lawall
The semantic patch that fixes this problem is as follows: (http://coccinelle.lip6.fr/) // <smpl> @r@ type T; identifier f; @@ static T f (...) { ... } @@ identifier r.f; declarer name EXPORT_SYMBOL_GPL; @@ -EXPORT_SYMBOL_GPL(f); // </smpl> Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-12crypto: arm - add support for GHASH using ARMv8 Crypto ExtensionsArd Biesheuvel
This implements the GHASH hash algorithm (as used by the GCM AEAD chaining mode) using the AArch32 version of the 64x64 to 128 bit polynomial multiplication instruction (vmull.p64) that is part of the ARMv8 Crypto Extensions. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-12crypto: arm - AES in ECB/CBC/CTR/XTS modes using ARMv8 Crypto ExtensionsArd Biesheuvel
This implements the ECB, CBC, CTR and XTS asynchronous block ciphers using the AArch32 versions of the ARMv8 Crypto Extensions for AES. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-12crypto: arm - add support for SHA-224/256 using ARMv8 Crypto ExtensionsArd Biesheuvel
This implements the SHA-224/256 secure hash algorithm using the AArch32 versions of the ARMv8 Crypto Extensions for SHA2. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-12crypto: arm - add support for SHA1 using ARMv8 Crypto InstructionsArd Biesheuvel
This implements the SHA1 secure hash algorithm using the AArch32 versions of the ARMv8 Crypto Extensions for SHA1. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-12crypto: arm - move ARM specific Kconfig definitions to a dedicated fileArd Biesheuvel
This moves all Kconfig symbols defined in crypto/Kconfig that depend on CONFIG_ARM to a dedicated Kconfig file in arch/arm/crypto, which is where the code that implements those features resides as well. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-10crypto: octeon - add SHA512 moduleAaro Koskinen
Add OCTEON SHA512 module. Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-10crypto: octeon - add SHA256 moduleAaro Koskinen
Add OCTEON SHA256 module. Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-10crypto: octeon - add SHA1 moduleAaro Koskinen
Add OCTEON SHA1 module. Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-10crypto: octeon - add instruction definitions for SHA1/256/512Aaro Koskinen
Add instruction definitions for SHA1/256/512. Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-10crypto: octeon - always disable preemption when using crypto engineAaro Koskinen
Always disable preemption on behalf of the drivers when crypto engine is taken into use. This will simplify the usage. Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-10crypto: octeon - don't disable bottom half in octeon-md5Aaro Koskinen
Don't disable bottom half while the crypto engine is in use, as it should be unnecessary: All kernel crypto engine usage is wrapped with crypto engine state save/restore, so if we get interrupted by softirq that uses crypto they should save and restore our context. This actually fixes an issue when running OCTEON MD5 with interrupts disabled (tcrypt mode=302). There's a WARNING because the module is trying to enable the bottom half with irqs disabled: [ 52.656610] ------------[ cut here ]------------ [ 52.661439] WARNING: CPU: 1 PID: 428 at /home/aaro/git/linux/kernel/softirq.c:150 __local_bh_enable_ip+0x9c/0xd8() [ 52.671780] Modules linked in: tcrypt(+) [...] [ 52.763539] [<ffffffff8114082c>] warn_slowpath_common+0x94/0xd8 [ 52.769465] [<ffffffff81144614>] __local_bh_enable_ip+0x9c/0xd8 [ 52.775390] [<ffffffff81119574>] octeon_md5_final+0x12c/0x1e8 [ 52.781144] [<ffffffff81337050>] shash_compat_digest+0xd0/0x1b0 Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-09crypto: powerpc - move files to fix build errorKim Phillips
The current cryptodev-2.6 tree commits: d9850fc529ef ("crypto: powerpc/sha1 - kernel config") 50ba29aaa7b0 ("crypto: powerpc/sha1 - glue") failed to properly place files under arch/powerpc/crypto, which leads to build errors: make[1]: *** No rule to make target 'arch/powerpc/crypto/sha1-spe-asm.o', needed by 'arch/powerpc/crypto/sha1-ppc-spe.o'. Stop. make[1]: *** No rule to make target 'arch/powerpc/crypto/sha1_spe_glue.o', needed by 'arch/powerpc/crypto/sha1-ppc-spe.o'. Stop. Makefile:947: recipe for target 'arch/powerpc/crypto' failed Move the two sha1 spe files under crypto/, and whilst there, rename other powerpc crypto files with underscores to use dashes for consistency. Cc: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Kim Phillips <kim.phillips@freescale.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-04crypto: powerpc/md5 - kernel configMarkus Stockhausen
Integrate the module into the kernel config tree. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-04crypto: powerpc/md5 - glueMarkus Stockhausen
Glue code for crypto infrastructure. Call the assembler code where required. Take a little care about small input data. Kick out early for input chunks < 64 bytes and replace memset for context cleanup with simple loop. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-04crypto: powerpc/md5 - assemblerMarkus Stockhausen
This is the assembler code for the MD5 implementation. Handling of algorithm constants has been slightly changed to reduce register usage and make better use of cores with multiple ALUs. Thus they are stored as delta values. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-02crypto: powerpc/sha1 - kernel configMarkus Stockhausen
Integrate the module into the kernel config tree. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-02crypto: powerpc/sha1 - glueMarkus Stockhausen
Glue code for crypto infrastructure. Call the assembler code where required. Disable preemption during calculation and enable SPE instructions in the kernel prior to the call. Avoid to disable preemption for too long. Take a little care about small input data. Kick out early for input chunks < 64 bytes and replace memset for context cleanup with simple loop. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-02crypto: powerpc/sha1 - assemblerMarkus Stockhausen
This is the assembler code for SHA1 implementation with the SIMD SPE instruction set. With the enhanced instruction set we can operate on 2 32 bit words in parallel. That helps reducing the time to calculate W16-W79. For increasing performance even more the assembler function can compute hashes for more than one 64 byte input block. The state of the used SPE registers is preserved via the stack so we can run from interrupt context. There might be the case that we interrupt ourselves and push sensitive data from another context onto our stack. Clear this area in the stack afterwards to avoid information leakage. The code is endian independant. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01crypto: powerpc/aes - kernel configMarkus Stockhausen
Integrate the module into the kernel configuration Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01cyprot: powerpc/aes - glue codeMarkus Stockhausen
Integrate the assembler modules into the kernel crypto framework. Take care to avoid long intervals of disabled preemption. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01crypto: powerpc/aes - ECB/CBC/CTR/XTS modesMarkus Stockhausen
The assembler block cipher module that controls the core AES functions. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01crypto: powerpc/aes - key handlingMarkus Stockhausen
Key generation for big endian core routines. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01crypto: powerpc/aes - assembler coreMarkus Stockhausen
The assembler AES encryption and decryption core routines. Implemented & optimized for big endian. Nevertheless they work on little endian too. For most efficient reuse in (higher level) block cipher routines they are implemented as "fast" call modules without any stack handling or register saving. The caller must take care of that part. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01crypto: powerpc/aes - aes tablesMarkus Stockhausen
4K AES tables for big endian. To reduce the possiblity of timing attacks, the size has been cut to 8KB + 256 bytes in contrast to 16KB in the generic implementation. That is not perfect but at least a good tradeoff for CPU limited router devices. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01crypto: powerpc/aes - register definesMarkus Stockhausen
Define some register aliases for better readability. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-03-01MIPS: BCM63xx: remove RSET_RNG register definitionsFlorian Fainelli
Now that these definitions have been moved to drivers/char/hw_random/bcm63xx-rng.c where they belong to make the driver standalone, we can safely remove these definitions from bcm63xx_regs.h. Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-02-28crypto: aesni - make driver-gcm-aes-aesni helper a proper aead algTadeusz Struk
Changed the __driver-gcm-aes-aesni to be a proper aead algorithm. This required a valid setkey and setauthsize functions to be added and also some changes to make sure that math context is not corrupted when the alg is used directly. Note that the __driver-gcm-aes-aesni should not be used directly by modules that can use it in interrupt context as we don't have a good fallback mechanism in this case. Signed-off-by: Adrian Hoban <adrian.hoban@intel.com> Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-02-27crypto: sha-mb - Fix big integer constant sparse warningLad, Prabhakar
this patch fixes following sparse warning: sha1_mb_mgr_init_avx2.c:59:31: warning: constant 0xF76543210 is so big it is long Signed-off-by: Lad, Prabhakar <prabhakar.csengg@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-02-27crypto: ppc/sha256 - kernel configMarkus Stockhausen
Integrate the module into the kernel config tree. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-02-27crypto: ppc/sha256 - glueMarkus Stockhausen
Glue code for crypto infrastructure. Call the assembler code where required. Disable preemption during calculation and enable SPE instructions in the kernel prior to the call. Avoid to disable preemption for too long. Take a little care about small input data. Kick out early for input chunks < 64 bytes and replace memset for context cleanup with simple loop. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2015-02-27crypto: ppc/sha256 - assemblerMarkus Stockhausen
This is the assembler code for SHA256 implementation with the SIMD SPE instruction set. Although being only a 32 bit architecture GPRs are extended to 64 bit presenting two 32 bit values. With the enhanced instruction set we can operate on them in parallel. That helps reducing the time to calculate W16-W64. For increasing performance even more the assembler function can compute hashes for more than one 64 byte input block. That saves a lot of register saving/restoring The state of the used SPE registers is preserved via the stack so we can run from interrupt context. There might be the case that we interrupt ourselves and push sensitive data from another context onto our stack. Clear this area in the stack afterwards to avoid information leakage. The code is endian independant. Signed-off-by: Markus Stockhausen <stockhausen@collogia.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
generated by cgit (git 2.34.1) at 2024-06-19 20:11:45 +0000