Page MenuHome GnuPG

libgcrypt rijndael self tests trigger bus error on solaris 10
Closed, ResolvedPublic


The self test fails under solaris 10. Libgcrypt 1.2.4 works fine.

my$ ./basic --debug
Starting Cipher checks.

checking BLOWFISH [4]
checking DES [302]
checking 3DES [2]
checking CAST5 [3]
checking AES [7]

Bus Error

For more info, see:

It appears to be an alignment issue.




Related Objects

Event Timeline

jas set Version to 1.4.1.
jas added a subscriber: jas.

I did some more code staring and comparing to 1.2.x without a result.
I have still no Solaris box available but the Debian build logs of 1.4.1 show
that the tests all run fine.

Next time I do code staring I should put off my Joo Janta 200 Super-Chromatic
Bug Sensitive Sunglasses first.

There are indeed a couple of non aligned accesses. Fixed in SVN trunk 1345.

If you make a release candidate I can test it on a solaris machine.

is the latest snapshot. I'd appreciate if you can test it so that I can close
this bug.

It doesn't seem to work:

my$ ../libtool --mode=execute gdb basic
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "sparc-sun-solaris2.10"...
(gdb) r --debug
Starting program: /tmp/jas/libgcrypt-1.4.4-svn1350/tests/.libs/basic --debug
Starting Cipher checks.

checking BLOWFISH [4]
checking DES [302]
checking 3DES [2]
checking CAST5 [3]
checking AES [7]

Program received signal SIGSEGV, Segmentation fault.
0xff32c99c in do_decrypt (ctx=0xffbe01d8,

bx=0xffbe01c8 "gC��Q\232���\232x�\t�\021�", 
ax=0xffbe01c8 "gC��Q\232���\232x�\t�\021�") at rijndael.c:289

289 *((u32*)w) = *((u32*)U1[w[0]]) ^ *((u32*)U2[w[1]])
(gdb) bt
#0 0xff32c99c in do_decrypt (ctx=0xffbe01d8,

bx=0xffbe01c8 "gC��Q\232���\232x�\t�\021�", 
ax=0xffbe01c8 "gC��Q\232���\232x�\t�\021�") at rijndael.c:289

#1 0xff32cf40 in rijndael_decrypt (context=0xffbe01d8,

b=0xffbe01c8 "gC��Q\232���\232x�\t�\021�", 
a=0xffbe01c8 "gC��Q\232���\232x�\t�\021�") at rijndael.c:708

#2 0xff32d5f8 in selftest_basic_128 () at rijndael.c:839
#3 0xff32d170 in rijndael_setkey (context=0x56b28,

key=0xffbe0588 "0123456789abcdef.,;/[]{}-=ABCDEF", keylen=16)
at rijndael.c:921

#4 0xff31750c in cipher_setkey (c=0x56188,

key=0xffbe0588 "0123456789abcdef.,;/[]{}-=ABCDEF", keylen=16)
at cipher.c:872

#5 0x00011a54 in check_one_cipher (algo=7, mode=1, flags=0) at basic.c:927
#6 0x00011d28 in check_ciphers () at basic.c:1057
#7 0x00013098 in main (argc=0, argv=0xffbff22c) at basic.c:2111

Btw, mingw32 needs this patch against tests/fipsdrv.c to avoid O_BINARY error.


include <fcntl.h> /* We need setmode(). */


Are you sure that the backtrace is really from svn1350? The given line numbers
don't match.

I'm relatively certain that it is correct. I created by running 'make dist' on
my machine and transferring the entire libgcrypt-*.tar.gz archive to the Solaris
system and building libgpg-error and the libgcrypt archive there. The system
doesn't have libgpg-error or libgcrypt installed before. The only way I see
them being wrong would be if there is a gdb/gcc problem, which isn't completely
unlikely, the tools on that platform are fairly old (gcc 3.4.4).

However, I re-made the step using today's svn version 1371 and it works fine!
So either it is now solved, or it only happens randomly. I'll re-run the
self-test a few times to check.

Btw, I get another warning during the self tests:

libgcrypt selftest: digest HMAC-SHA256 (8): does not match in second
implementation (data-28 key-4)
libgcrypt selftest: digest HMAC-SHA256 (8): Selftest failed

The self-test passes anyway though.


I've re-run the 'basic' self test several time, and it works every time (except
for the hmac problem).


Out of curiosity, I tried to rebuild the 1350 revision that you linked to below,
and it also worked. I'm not sure what happened, but at least I cannot reproduce
this any more.


Sometimes weird things happen :_(.

I did no change to the aes code since 1350 and I dubt that tehre is a way to get
a segv from the selftest. The original problem with the aes code was a sigbus.

I'll check what's going on with HMAC.

The second implementaion hmac code is also used by the hmac256 utiliy we install
since some time. Can you please run this and compare the result? (best with the
hmac256 in libgcrypt/src/):

$ printf "what do ya want for nothing?" | hmac256 Jefe

my$ printf "what do ya want for nothing?" | ./hmac256 Jefe
hmac256: fatal error: self-test failed

werner removed a project: Restricted Project.
werner removed a project: In Progress.
werner claimed this task.
werner added a project: Duplicate.

Fixed. The new hmac256 problem has been split out to T979.