Fix incorrect counter overflow handling for GCM
* cipher/cipher-gcm.c (gcm_ctr_encrypt): New function to handle 32-bit CTR increment for GCM. (_gcry_cipher_gcm_encrypt, _gcry_cipher_gcm_decrypt): Do not use generic CTR implementation directly, use gcm_ctr_encrypt instead. * tests/basic.c (_check_gcm_cipher): Add test-vectors for 32-bit CTR overflow. (check_gcm_cipher): Add 'split input to 15 bytes and 17 bytes' test-runs.
I believe we have found what seems to be a bug in counter overflow
handling in AES-GCM in libgcrypt's implementation. This leads to
incorrect results when using a non-12-byte IV and decrypting payloads
encrypted with other AES-GCM implementations, such as OpenSSL.
According to the NIST Special Publication 800-38D "Recommendation for
Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC",
section 7.1, algorithm 4, step 3 [NIST38D], the counter increment is
defined as inc_32. Section 6.2 of the same document defines the
incrementing function inc_s for positive integers s as follows:
the function increments the right-most s bits of the string, regarded as the binary representation of an integer, modulo 2^s; the remaining, left-most len(X) - s bits remain unchanged
(X is the complete counter value in this case)
This problem does not occur when using a 12-byte IV, because AES-GCM has
a special case for the inital counter value with 12-byte IVs:
If len(IV)=96, then J_0 = IV 0^31 1
i.e., one would have to encrypt (UINT_MAX - 1) * blocksize of data to
hit an overflow. However, for non-12-byte IVs, the initial counter value
is the output of a hash function, which makes hitting an overflow much
In practice, we have found that usingiv = 9e 79 18 8c ff 09 56 1e c9 90 99 cc 6d 5d f6 d3 key = 26 56 e5 73 76 03 c6 95 0d 22 07 31 5d 32 5c 6b a5 54 5f 40 23 98 60 f6 f7 06 6f 7a 4f c2 ca 40
will reliably trigger an overflow when encrypting 10 MiB of data. It
seems that this is caused by re-using the AES-CTR implementation for
incrementing the counter.
Bug was introduced by commit bd4bd23a2511a4bce63c3217cca0d4ecf0c79532
"GCM: Use counter mode code for speed-up".
- Reported-by: Clemens Lang <Clemens.Lang@bmw.de>
- GnuPG-bug-id: T3764
- Signed-off-by: Jussi Kivilinna <email@example.com>