tests fail in 2021
Closed, ResolvedPublic

Description

While working on the reproducible builds project for openSUSE, I rebuild all packages with a time set to 15 years in the future, to ensure that build results do not differ to what it is today.

I found that in 2033, for gpgme 2 tests fail

one is trivial to fix:

lang/python/tests/t-callbacks.py
-Expire-Date: 2020-12-31
+Expire-Date: 2999-12-31

but the other issue can probably be solved better than this:

+++ gpgme-1.10.0/lang/qt/tests/t-tofuinfo.cpp
@@ -455,7 +455,7 @@ private Q_SLOTS:
 
         QVERIFY(result.numSignatures() == 1);
         auto sig = result.signatures()[0];
-        QVERIFY(sig.validity() == Signature::Marginal); 
+        QVERIFY(sig.validity() == Signature::Marginal || sig.validity() == Signature::Unknown);
 
         auto stats = sig.key().userID(0).tofuInfo();
         QVERIFY(!stats.isNull());
werner added a subscriber: werner.

Can you please show the output of these failing tests? I assume you are running on a 64 bit platform.

Setting the expiration date to 2999 is not a good idea because OpenPGP works only until 2106. But we need to see what really failed. The second test might be related to this.

Here is the build log from unpatched gpgme https://www.zq1.de/~bernhard/temp/gpgme-build-log-2033.txt
it has some tracebacks from t-callbacks.py

Build host is kvm on x86_64

The 2nd part also fails with expiry moved to 2037. Then it shows such messages:

QT_HASH_SEED: forced seed value is not 0, cannot guarantee that the hashing functions will produce a stable value.********* Start testing of TofuInfoTest *********
FAIL!  : TofuInfoTest::testTofuConflict() 'sig.validity() == Signature::Marginal' returned FALSE. ()
Totals: 7 passed, 1 failed, 0 skipped, 0 blacklisted, 2009ms

I currently do export QT_HASH_SEED=42 in the build environment - that helped with QHash randomness in other places.

werner added a project: qt.Feb 28 2018, 8:34 AM
werner added a subscriber: aheinecke.
werner assigned this task to BenM.Apr 17 2018, 8:29 PM

Ben: We need to use a faked system time thing to make those tests more stable.

werner triaged this task as Normal priority.Apr 17 2018, 8:30 PM
BenM added a comment.Apr 26 2018, 6:44 AM

Right.

Not to mention making sure we test for a time after the end of the old 32-bit clock.

So that's after Tue Jan 19 03:14:07 2038 UTC (aka 2038-01-19T03:14:07Z).

I'd hope nothing still on the market could be tripped up by something like that, but who knows what's out there and still ticking.

BenM added a comment.May 5 2018, 5:10 AM

The Python portion of this is done, the tests will now create a key with an expiration a few years shy of the 2106 end date (NYE 2099).

That leaves the qt issue to be done.

BenM closed this task as Resolved.Dec 10 2018, 6:19 AM

Though apparently resolved back in May, this is what ultimately led to T4191 and was thus only properly resolved quite recently.

hedning added a subscriber: hedning.Jan 8 2019, 8:05 PM

We've run into the testTofuConflict failure on NixOS. gpgme v1.12, gnupg v2.2.12.

Build log: https://hydra.nixos.org/build/86830084/nixlog/1

Not sure if it's actually the same issue, but it's the exact same test at least:

FAIL!  : TofuInfoTest::testTofuConflict() 'sig.validity() == Signature::Marginal' returned FALSE. ()
   Loc: [t-tofuinfo.cpp(458)]
werner reopened this task as Open.Jan 9 2019, 8:00 AM

I don't know why @BenM closed this bug given that he mentioned that the qt part is yet not solved.

aheinecke claimed this task.Jan 9 2019, 8:12 AM
aheinecke added a subscriber: BenM.

Oh,.. it is even worse. The conflict keys expired 2019-01-06 so they are actually expired right now.

I'll change the test to use keys without expiry.

aheinecke closed this task as Resolved.Jan 9 2019, 8:52 AM

I sent a message to gnupg-devel about this issue as it will probably hit more people now that the keys used are expired :-(