Page MenuHome GnuPG

gnupg2 test failure on s390x
Testing, NormalPublic

Description

The gnupg2 tests are failing on s390x with the new libgcrypt:

    > () (- - -) (default default never) (rsa sign auth encr seconds=600) (rsa auth,encr 2) (rsa sign 2038-01-01) (rsa sign 20380101T115500) (rsa sign 2d) (rsa1024 sign 2w) (rsa2048 encr 2m) (rsa4096 sign,auth 2y) ("/builddir/build/BUILD/gnupg-2.2.27/g10/gpg" --no-permission-warning --always-trust --quick-add-key "C0E2945C131F3D65DBCE8B031EC5E7B1A792A6C1" rsa4096 sign,auth "2y") failed: gpg: signing failed: Bad passphrase
gpg: make_keysig_packet failed for backsig: Bad passphrase
gpg: Key generation failed: Bad passphrase
gpg: Key not changed so no update needed.
0: tests.scm:122: (throw (string-append (stringify what) " failed") (:stderr result))
1: quick-key-manipulation.scm:134: (call-check `(,@gpg --quick-add-key ,fpr ,@args))
2: #<CLOSURE>
3: tests.scm:78: (apply proc args)
4: #<CLOSURE>
5: init.scm:230: (apply proc cars)
FAIL: tests/openpgp/quick-key-manipulation.scm

Disabling the s390x-msa HW acceleration makes everything pass as expected so I suspect the issue will be somewhere down in the new accelerated code. We hit it first time in Fedora rawhide rebuild with new gcc11, but I can reproduce it also on Fedora 32 with older gcc10. The failed build is available here (without the logs now, but I can provide them if needed):

https://koji.fedoraproject.org/koji/taskinfo?taskID=62220845

I debugged this issue up to the place where the AES OCB decryption tag is not matching the expected value and I was able to "skip" the tag verification using gdb to verify that the above test case passes (the decoded data are usable) so I assume there is some issue in the tag creation, but that is probably the end of my findings as I am not experienced with s390x asembler. I do not know why it was not caught by the libgcrypt testsuite either.

Let me know if there is some more information I can provide to help with figuring out the issue.

Details

Version
libgcrypt-1.9.2;gnupg-2.2.27

Event Timeline

I did a bit digging and it looks like the code path using accelerator is not hit because the test vecors have max ~48 blocks, but accelerator is involved only with 64 blocks and more if I read the code right. So we need 1) larger test vector to invoke this code path in libgcrypt 2) figure out what goes wrong there.

Let me know if there is something I can help with.

I have a minimal reproducer:

diff --git a/tests/basic.c b/tests/basic.c
index 9a7e33cc..73ae01db 100644
--- a/tests/basic.c
+++ b/tests/basic.c
@@ -6346,11 +6346,152 @@ do_check_ocb_cipher (int inplace)
       "033ac4d13c3decc4c62d7de718ace802"
       "140452dc850989f6762e3578bbb04be3"
       "1a237c599c4649f4e586b2de"
+    },
+    { GCRY_CIPHER_AES, 12, "0F0E0D0C0B0A09080706050403020100",
+      "BBAA9988776655443322110D",
+      "000102030405060708090A0B0C0D0E0F1011121314151617"
+      "18191A1B1C1D1E1F2021222324252627",
+      /* test vector for checksumming */
+      "01000000000000000000000000000000"
+      "02000000000000000000000000000000"
+      "04000000000000000000000000000000"
+      "08000000000000000000000000000000"
+      "10000000000000000000000000000000"
+      "20000000000000000000000000000000"
+      "40000000000000000000000000000000"
+      "80000000000000000000000000000000"
+      "00010000000000000000000000000000"
+      "00020000000000000000000000000000"
+      "00040000000000000000000000000000"
+      "00080000000000000000000000000000"
+      "00100000000000000000000000000000"
+      "00200000000000000000000000000000"
+      "00400000000000000000000000000000"
+      "00800000000000000000000000000000"
+      "00000100000000000000000000000000"
+      "00000200000000000000000000000000"
+      "00000400000000000000000000000000"
+      "00000800000000000000000000000000"
+      "00001000000000000000000000000000"
+      "00002000000000000000000000000000"
+      "00004000000000000000000000000000"
+      "00008000000000000000000000000000"
+      "00000001000000000000000000000000"
+      "00000002000000000000000000000000"
+      "00000004000000000000000000000000"
+      "00000008000000000000000000000000"
+      "00000010000000000000000000000000"
+      "00000020000000000000000000000000"
+      "00000040000000000000000000000000"
+      "00000080000000000000000000000000"
+      "00000000010000000000000000000000"
+      "00000000020000000000000000000000"
+      "00000000040000000000000000000000"
+      "00000000080000000000000000000000"
+      "00000000100000000000000000000000"
+      "00000000200000000000000000000000"
+      "00000000400000000000000000000000"
+      "00000000800000000000000000000000"
+      "00000000000100000000000000000000"
+      "00000000000200000000000000000000"
+      "00000000000400000000000000000000"
+      "00000000000800000000000000000000"
+      "00000000001000000000000000000000"
+      "00000000002000000000000000000000"
+      "00000000004000000000000000000000"
+      "00000000008000000000000000000000"
+      "02000000000000000000000000000000"
+      "04000000000000000000000000000000"
+      "08000000000000000000000000000000"
+      "10000000000000000000000000000000"
+      "20000000000000000000000000000000"
+      "40000000000000000000000000000000"
+      "80000000000000000000000000000000"
+      "00010000000000000000000000000000"
+      "00020000000000000000000000000000"
+      "00040000000000000000000000000000"
+      "00080000000000000000000000000000"
+      "00100000000000000000000000000000"
+      "00200000000000000000000000000000"
+      "00400000000000000000000000000000"
+      "00800000000000000000000000000000"
+      "00000100000000000000000000000000"
+      "00000200000000000000000000000000"
+      "00000400000000000000000000000000"
+      "00000800000000000000000000000000",
+      "01105c6e36f6ac480f022c51e31ed702"
+      "90fda4b7b783194d4b4be8e4e1e2dff4"
+      "6a0804d1c5f9f808ea7933e31c063233"
+      "2bf65a22b20bb13cde3b80b3682ba965"
+      "b1207c58916f7856fa9968b410e50dee"
+      "98b35c071163d1b352b9bbccd09fde29"
+      "b850f40e71a8ae7d2e2d577f5ee39c46"
+      "7fa28130b50a123c29958e4665dda9a5"
+      "e0793997f8f19633a96392141d6e0e88"
+      "77850ed4364065d1d2f8746e2f1d5fd1"
+      "996cdde03215306503a30e41f58ef3c4"
+      "400365cfea4fa6381157c12a46598edf"
+      "18604854462ec66e3d3cf26d4723cb6a"
+      "9d801095048086a606fdb9192760889b"
+      "a8ce2e70e1b55a469137a9e2e6734565"
+      "283cb1e2c74f37e0854d03e33f8ba499"
+      "ef5d9af4edfce077c6280338f0a64286"
+      "2e6bc27ebd5a4c91b3778e22631251c8"
+      "c5bb75a10945597a9d6c274fc82d3338"
+      "b403a0a549d1375f26e71ef22bce0941"
+      "93ea87e2ed72fce0546148c351eec3be"
+      "867bb1b96070c377fff3c98e21562beb"
+      "475cfe28abcaaedf49981f6599b15140"
+      "ea6130d24407079f18ba9d4a8960b082"
+      "b39c57320e2e064f02fde88c23112146"
+      "1cac3655868aef584714826ee4f361fb"
+      "e6d692e1589cbb9dd3c74fa628df2a1f"
+      "3b0029b1d62b7e9978013ed3c793c1dd"
+      "1f184c8f7022a853cac40b74ac749aa3"
+      "f33f0d14732dfda0f2c3c20591bf1f5a"
+      "710ec0d0bca342baa5146068a78ff58c"
+      "66316312b7a98af35a0f4e92799b4047"
+      "f047ae61f25c28d232ce5c168cc745d6"
+      "6da13cb0f9e38a696635dba7a21571cf"
+      "cd64ec8cc33db7879f59a90d9edd00f6"
+      "a899e39ab36b9269a3ac04ebad9326bf"
+      "53cd9b400168a61714cd628a4056d236"
+      "bd8622c76daa54cb65f5db2fe03bafbe"
+      "0b23549ae31136f607293e8093a21934"
+      "74fd5e9c2451b4c8e0499e6ad34fafc8"
+      "ab77722a282f7f84b14ddebf7e696300"
+      "c1ef92d4a0263c6cca104530f996e272"
+      "f58992ff68d642b071a5848dc4acf2ae"
+      "28fb1f27ae0f297d5136a7a0a4a03e89"
+      "b588755b8217a1c62773790e69261269"
+      "19f45daf7b3ccf18e3fc590a9a0e172f"
+      "033ac4d13c3decc4c62d7de718ace802"
+      "140452dc850989f6762e3578bbb04be3"
+      "a8ae66427697167e85725b37b304baf0"
+      "56dbcef79fbb97cdfe1590e5f3d0bd1b"
+      "ce518f2f141960a1c80a4fe787b90b63"
+      "e7b0e0a0d8d522619130c544bb1abad0"
+      "b267c650e8916b5d7ececfeea7f0ad15"
+      "206a92581319946b138764f209109a20"
+      "0146b4cfb2ce8bd0db5c2cd5b495c56f"
+      "8f8a7934fe1f9add0674d4549080bf0d"
+      "01149ed18dbdccc5e54a3e7039546970"
+      "401ecc885902ee3dcfad504a68066f92"
+      "c779f1e1c48d37ba0e177ac652c1827b"
+      "f1f6723d533f0cdf36331e3ad1e1b1af"
+      "bc89a29c87fe3603353130d0dfbe1f29"
+      "13ad144e7c6515fb92005b6ece218b4f"
+      "baedc42d484fffee39df88041b49342a"
+      "6134cc7ca46d40d274c1ffafa98956e6"
+      "a492486989c4e328761c01798abcb09b"
+      "a42eb115334619daaeae9175f365fe9f"
+      "e5c3b254379d546005016784015f729f"
+      "4715ff6db16c5d16333e03fd"
     }
   };
   gpg_error_t err = 0;
   gcry_cipher_hd_t hde, hdd;
-  unsigned char out[1024];
+  unsigned char out[2048];
   unsigned char tag[16];
   int tidx;

Normally, it passes, but with HW acceleration on s390x I can notice this failure:

basic: cipher-ocb, gcry_cipher_checktag failed (tv 18): Checksum error
expected: 47 15 ff 6d b1 6c 5d 16 33 3e 03 fd
computed: b9 64 32 e5 8f d6 ae ea 11 73 9a 78
basic: cipher-ocb, decrypt tag mismatch (tv 18)

The values are reproducible.

Thanks for the report.

There is test in tests/basic that should have gotten this but did not - check_ocb_cipher_largebuf_split.

That function passes 2+MiB buffer to OCB and check generated tag against expected value. Bug in that test is that buffer is initialized with

for (i = 0; i < buflen; i++)
  inbuf[i] = (unsigned int)(i + 181081) * 5039U;

and values repeat after every 256 bytes or 16 blocks. First part of OCB tag calculation is xoring plaintext blocks together, so it repeating patterns of plaintext get cancelling each out from end result.

Then when I implemented OCB for s390x, I expected check_ocb_cipher_largebuf_split to work - and expected check_ocb_cipher_checksum to work but apparently it only checked encryption routine.

Here's patches for tests/basic and rijndael-s390x:



Thanks! Tested the above patches and now all the tests pass on the machine where I saw the failures.

The changes look good to me. Can we expect bugfix release in close future or should I just pull them into Fedora for now to be able to build new gnupg?

@werner Can you comment about bugfix release?

I already backported the above for Fedora so I am not in hurry now. But I believe others might hit the same issue.

We have two or three other open issue which I would like to address before a release. FWIW, release ticket is T5305.

werner changed the task status from Open to Testing.Mar 30 2021, 5:41 PM
werner triaged this task as Normal priority.Apr 15 2021, 9:03 AM