gpgme python bindings test t-callbacks.py fails on 32-bit platforms
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	dkg
	Oct 18 2018, 6:38 AM

Description

the debian build daemon logs of gpgme 1.12.0 show that the 32-bit architectures are failing, but the 64-bit architectures are successful.

Looking at the armhf and i386 buildd logs, it appears that the problem is with t-callbacks.py in the python bindings.

one example error message is:

make[5]: Leaving directory '/<<PKGBUILDDIR>>/lang/python/tests'
GNUPGHOME=/<<PKGBUILDDIR>>/lang/python/tests LC_ALL=C GPG_AGENT_INFO= top_srcdir=../../.. srcdir=. LD_LIBRARY_PATH="../../../src/.libs:" /usr/bin/python2 ./run-tests.py \
  --interpreters="/usr/bin/python2 /usr/bin/python3" --srcdir=.  \
  initial.py t-wrapper.py t-callbacks.py t-data.py t-encrypt.py t-encrypt-sym.py t-encrypt-sign.py t-sign.py t-signers.py t-decrypt.py t-verify.py t-decrypt-verify.py t-sig-notation.py t-export.py t-import.py t-trustlist.py t-edit.py t-keylist.py t-keylist-from-data.py t-wait.py t-encrypt-large.py t-file-name.py t-idiomatic.py t-protocol-assuan.py t-quick-key-creation.py t-quick-subkey-creation.py t-quick-key-manipulation.py t-quick-key-signing.py final.py
starting gpg-agent..
OK
Using gpg module from '/<<PKGBUILDDIR>>/lang/python/python2-gpg/lib.linux-i686-2.7/gpg'.
Traceback (most recent call last):
  File "./t-callbacks.py", line 114, in <module>
    c.op_genkey(parms, None, None)
  File "/<<PKGBUILDDIR>>/lang/python/python2-gpg/lib.linux-i686-2.7/gpg/core.py", line 158, in wrapper
    return _funcwrap(self, *args)
  File "/<<PKGBUILDDIR>>/lang/python/python2-gpg/lib.linux-i686-2.7/gpg/core.py", line 141, in _funcwrap
    return errorcheck(result, name)
  File "/<<PKGBUILDDIR>>/lang/python/python2-gpg/lib.linux-i686-2.7/gpg/errors.py", line 129, in errorcheck
    raise GPGMEError(retval, extradata)
gpg.errors.GPGMEError: gpgme_op_genkey: GPGME: General error
stopping gpg-agent 
starting gpg-agent..
gpg-connect-agent: no running gpg-agent - starting '/usr/bin/gpg-agent|--debug-quick-random'
gpg-connect-agent: waiting for the agent to come up ... (5s)
gpg-connect-agent: connection to agent established
OK
Using gpg module from '/<<PKGBUILDDIR>>/lang/python/python3-gpg/lib.linux-i686-3.6/gpg'.
Traceback (most recent call last):
  File "./t-callbacks.py", line 114, in <module>
    c.op_genkey(parms, None, None)
  File "/<<PKGBUILDDIR>>/lang/python/python3-gpg/lib.linux-i686-3.6/gpg/core.py", line 158, in wrapper
    return _funcwrap(self, *args)
  File "/<<PKGBUILDDIR>>/lang/python/python3-gpg/lib.linux-i686-3.6/gpg/core.py", line 141, in _funcwrap
    return errorcheck(result, name)
  File "/<<PKGBUILDDIR>>/lang/python/python3-gpg/lib.linux-i686-3.6/gpg/errors.py", line 129, in errorcheck
    raise GPGMEError(retval, extradata)
gpg.errors.GPGMEError: gpgme_op_genkey: GPGME: General error
stopping gpg-agent

Details

Version: 1.12.0

Related Objects

Mentioned In: T3815: tests fail in 2021
rM7c63bfe4ab43: python: docs updates
Mentioned Here: T4192: python gpg segfaults when first parameter to gpg.Context().op_genkey() is None or otherwise malformed
T4195: Fix time API in gpgme
rM46da79e3de99: python: key expiration datetime stamp tests

Event Timeline

dkg created this task.Oct 18 2018, 6:38 AM

dkg updated the task description. (Show Details)

• werner triaged this task as High priority.Oct 18 2018, 11:46 AM

• werner added a project: Python.

here's me running just the specific test:

 dkg@unstable-i386:~/src/gpgme/gpgme$ GPGME_DEBUG=9:$(pwd)/gpgme_debug.log LD_LIBRARY_PATH=$(pwd)/build/src/.libs PYTHONPATH=$(pwd)/build/lang/python/python3.6-gpg/lib.linux-i686-3.6  python3 lang/python/tests/t-callbacks.py 
Traceback (most recent call last):
  File "lang/python/tests/t-callbacks.py", line 114, in <module>
    c.op_genkey(parms, None, None)
  File "/home/dkg/src/gpgme/gpgme/build/lang/python/python3.6-gpg/lib.linux-i686-3.6/gpg/core.py", line 158, in wrapper
    return _funcwrap(self, *args)
  File "/home/dkg/src/gpgme/gpgme/build/lang/python/python3.6-gpg/lib.linux-i686-3.6/gpg/core.py", line 141, in _funcwrap
    return errorcheck(result, name)
  File "/home/dkg/src/gpgme/gpgme/build/lang/python/python3.6-gpg/lib.linux-i686-3.6/gpg/errors.py", line 129, in errorcheck
    raise GPGMEError(retval, extradata)
gpg.errors.GPGMEError: gpgme_op_genkey: GPGME: General error
1 dkg@unstable-i386:~/src/gpgme/gpgme$

the generated debug log is here:

gpgme_debug.log.txt156 KBDownload

(edited because earlier run was pointing at the wrong libraries)

The error might have to do with rM46da79e3de99a7b65748994921d6aab73b9974e7 which looks like it might run afoul of 32-bit time_t (Y2K38 problem?).

if i revert it, then the test suite passes.

I propose changing that expiration date to be 2037-12-31 instead of 2099.

But we really need to address the Y2K38 problem for gpgme at some point soon (i don't know how)

I have not looked at the new error but the year 2099 is clearly a y2k38 problem. gpg has some precautions but I have not checked the key generation code. The gpgme interface uses a signed long for the expiration time, although that it parses the dates received from gpg as an unsigned long. Right now, I am not sure why we did this because an unsigned long would just work. Maybe we can change or enhance the interface. But in any case this is a general problem and not specific to this bug.

See T4195 for the general problem

the error i'd seen earlier after reverting the change was an error due to running t-callbacks.py on its own, without the rest of the test suite framework. running it within the test suite framework (with the change reverted), it passes without a problem. I've uploaded 1.12.0-4 to debian with a patch to t-callbacks.py. I can apply it upstream, if you want me to.

That is up to @BenM

Well, I guess this answers my question in T4192 regarding why op_genkey was in use.

The t-callbacks.py test probably pre-dates the ctx.create_key function, so that answers that.

As for the error caused by my old commit; I agree that that's the cause, though arguably the test has done its job properly by failing to pass.

It might be best in the interim to rewrite that test to check whether it's running on a 64-bit OS or a 32-bit one and expect different results depending on what it finds. Either that or put the 2099 check inside a try/except statement with 64-bit (pass) coming first and 32-bit (fail, try 2037 instead) on the exceptions. I'll look into it.

Nesting the op_genkey() calls inside try/except statements with the exceptions being caught as "oops" and otherwise "oops" being set to None provides a means of checking whether the 2099 expiration is a problem and 2037 is not.

It behaves as expected over here in 64-bit land, but presumably @dkg can check the ancient architecture for us in a new build.

BenM changed the task status from Open to Testing.Nov 22 2018, 10:31 AM

i'd be happy to help you set up your own x86 32-bit guest VM for testing
if you like, even if you're running on x86_64 hardware. they're cheap
and easy to run, and have a delightfully small memory footprint :P just
let me know!

I'll deploy one on AWS somewhere briefly once I've replaced a certain external keyboard, there will almost certainly be an existing image of some Linux distro in the AWS marketplace and I'd be very surprised if it took more than an hour or two of compute time to confirm.

This has now been tested on a 32-bit Gentoo VM and it behaves as expected with 32-bit system detection and creating keys with pre-2038 expirations working.

Note that gpg.Context().create_key() will still attempt to generate a key on a 32-bit system where the expiration is after January 19, 2038, but of course the expiration date is not handled well when that occurs.

Nevertheless, as this is all expected behaviour, closing this as fixed.

BenM mentioned this in rM7c63bfe4ab43: python: docs updates.Dec 10 2018, 6:12 AM

BenM mentioned this in T3815: tests fail in 2021.Dec 10 2018, 6:19 AM