Fix incorrect counter overflow handling for GCM

* cipher/cipher-gcm.c (gcm_ctr_encrypt): New function to handle 32-bit CTR increment for GCM. (_gcry_cipher_gcm_encrypt, _gcry_cipher_gcm_decrypt): Do not use generic CTR implementation directly, use gcm_ctr_encrypt instead. * tests/basic.c (_check_gcm_cipher): Add test-vectors for 32-bit CTR overflow. (check_gcm_cipher): Add 'split input to 15 bytes and 17 bytes' test-runs.

I believe we have found what seems to be a bug in counter overflow

handling in AES-GCM in libgcrypt's implementation. This leads to

incorrect results when using a non-12-byte IV and decrypting payloads

encrypted with other AES-GCM implementations, such as OpenSSL.According to the NIST Special Publication 800-38D "Recommendation for

Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC",

section 7.1, algorithm 4, step 3 [NIST38D], the counter increment is

defined as inc_32. Section 6.2 of the same document defines the

incrementing function inc_s for positive integers s as follows:

the function increments the right-most s bits of the string, regarded as the binary representation of an integer, modulo 2^s; the remaining, left-most len(X) - s bits remain unchanged (X is the complete counter value in this case)

This problem does not occur when using a 12-byte IV, because AES-GCM has

a special case for the inital counter value with 12-byte IVs:

If len(IV)=96, then J_0 = IV 0^31 1 i.e., one would have to encrypt (UINT_MAX - 1) * blocksize of data to

hit an overflow. However, for non-12-byte IVs, the initial counter value

is the output of a hash function, which makes hitting an overflow much

more likely.In practice, we have found that using

iv = 9e 79 18 8c ff 09 56 1e c9 90 99 cc 6d 5d f6 d3 key = 26 56 e5 73 76 03 c6 95 0d 22 07 31 5d 32 5c 6b a5 54 5f 40 23 98 60 f6 f7 06 6f 7a 4f c2 ca 40will reliably trigger an overflow when encrypting 10 MiB of data. It

seems that this is caused by re-using the AES-CTR implementation for

incrementing the counter.

Bug was introduced by commit bd4bd23a2511a4bce63c3217cca0d4ecf0c79532

"GCM: Use counter mode code for speed-up".

- Reported-by: Clemens Lang <Clemens.Lang@bmw.de>
- GnuPG-bug-id: T3764
- Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>