cipher-gcm-intel-pclmul.c:418:17: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'size_t' (aka 'unsigned long')
Closed, InvalidPublic

Description

Compiled libgcrypt 1.7.8 with bleading edge clang-5 ( 5.0.0 (trunk 305735)) and UBSan on Ubuntu 16.04 x64. Used the following flags:

-O2 -fno-omit-frame-pointer -g -fsanitize=address -fsanitize=undefined -fsanitize=integer -fsanitize-coverage=trace-pc-guard -fno-sanitize-recover=undefined -fsanitize=float-divide-by-zero -fsanitize=float-cast-overflow -fsanitize-undefined-trap-on-error -fno-sanitize-recover=all

Ran the following test:

libgcrypt-1.7.8/tests/basic
cipher-gcm-intel-pclmul.c:418:17: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'size_t' (aka 'unsigned long')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior cipher-gcm-intel-pclmul.c:418:17 in

Details

Version
1.7.8
geeknik created this task.Jul 4 2017, 3:10 AM
gniibe claimed this task.Jul 4 2017, 5:06 AM
gniibe closed this task as Invalid.
gniibe triaged this task as Low priority.
gniibe added a subscriber: gniibe.

See T3248: mpiutil.c:501:37: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned long' for unsigned integer overflow.

Even if it were undefined (except something like SEGV), the result of the value nblocks in the operation nblocks-- will not be used, because the value evaluated as nblocks-- is 0 and the loop exits and no use of nblocks any more.