keccak: Use size_t to avoid integer overflow
9ee2d56e806bUnpublished
Actions

Unpublished Commit · Learn More

Not On Permanent Ref: This commit is not an ancestor of any permanent ref.

Description

keccak: Use size_t to avoid integer overflow

* cipher/keccak-armv7-neon.S: Fix function name in comment and change
  parameter type to size_t.
* cipher/keccak.c (keccak_ops_t): Change absorb function signature to
  use size_t.
  (keccak_absorb_lanes64_avx512): Change nlanes type to size_t.
  (_gcry_keccak_absorb_lanes64_armv7_neon): Ditto.
  (keccak_absorb_lanes64_armv7_neon): Ditto.
  (keccak_absorb_lanes32bi): Ditto.
  (keccak_absorb_lanes32bi_bmi2): Ditto.
  (keccak_write): Change nlanes variable to use size_t and avoid
  overflow when calculating count.
* cipher/keccak_permute_64.h (KECCAK_F1600_ABSORB_FUNC_NAME): Change
  nlanes argument to use size_t.

Cherry-pick master commit of:
9c828129b2058c3f36e07634637929a54e8377ee

Any input to the SHA3 functions > 4GB was giving wrong result when it
was invoked in one-shot, while working correctly when it was fed by
chunks. It turned out that the calculation in the keccak_write
overflows the unsigned int type (nlanes * 8 does not fit 32b when
the inlen > 4GB).

GnuPG-bug-id: T6217
Signed-off-by: Jakub Jelen <jjelen@redhat.com>