mpiutil.c:501:37: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned long'
Closed, InvalidPublic

Description

Compiled libgcrypt 1.7.8 with bleading edge clang-5 ( 5.0.0 (trunk 305735)) and UBSan on Ubuntu 16.04 x64. Used the following flags:

-O2 -fno-omit-frame-pointer -g -fsanitize=address -fsanitize=undefined -fsanitize=integer -fsanitize-coverage=trace-pc-guard -fno-sanitize-recover=undefined -fsanitize=float-divide-by-zero -fsanitize=float-cast-overflow -fsanitize-undefined-trap-on-error -fno-sanitize-recover=all

Ran the following test:

libgcrypt-1.7.8/tests/fips186-dsa
mpiutil.c:501:37: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior mpiutil.c:501:37 in

geeknik created this task.Jul 4 2017, 3:11 AM
gniibe claimed this task.Jul 4 2017, 4:58 AM
gniibe closed this task as Invalid.
gniibe triaged this task as Low priority.
gniibe added a subscriber: gniibe.

It is intentionally used.
And in the C programming language, it is defined that unsigned integer never overflows (it is computed as modulo 2).