gpgme-1.8.0: test failures on NetBSD
Closed, ResolvedPublic

Description

In another bug report I was asked to test with gpgme-1.8.0, so I built
that and ran the self tests on NetBSD-7.99.67/amd64.

There are entirely too many errors for my taste:

===> Testing for gpgme-1.8.0
Making check in src
Making check in tests
Making check in gpg
/usr/bin/make  check-TESTS check-local
starting gpg-agent
PASS: initial.test
t-encrypt.c:63: GPGME: End of file
FAIL: t-encrypt
Begin Result Encryption:
-----BEGIN PGP MESSAGE-----

jA0EBwMCEARHrQ5k02tg0kEBEoto9H53XdEMugy6rzjebXTrX+pQJhZ4ulKCLXLK
qr0F7Yly3ta69+YO+19NyNOZUVzEP+lpgRVChofPEWf4zA==
=PsQN
-----END PGP MESSAGE-----
End Result.
Begin Result Decryption:
Hallo Leute
End Result.
PASS: t-encrypt-sym
t-encrypt-sign.c:119: GPGME: End of file
FAIL: t-encrypt-sign
Wrong fingerprint reported: ADAB7FCC1F4DE2616ECFA402AF82244F9CD9FD55
FAIL: t-sign
t-signers.c:126: GPGME: End of file
FAIL: t-signers
t-decrypt.c:69: GPGME: Decryption failed
FAIL: t-decrypt
t-verify.c:112: Unexpected signature summary: want=0x0 have=0x80
FAIL: t-verify
t-decrypt-verify.c:126: GPGME: Decryption failed
FAIL: t-decrypt-verify
PASS: t-sig-notation
Begin Result:
End Result.
t-export.c:74: GPGME: End of file
FAIL: t-export
PASS: t-import
PASS: t-trustlist
t-edit.c:140: GPGME: End of file
FAIL: t-edit
Primary key `Alfa Test' has unexpected key ID: AF82244F9CD9FD55
FAIL: t-keylist
Less keys returned than expected
FAIL: t-keylist-sig
PASS: t-wait
t-encrypt-large.c:127: GPGME: End of file
FAIL: t-encrypt-large
t-file-name.c:70: GPGME: End of file
FAIL: t-file-name
PASS: t-gpgconf
t-encrypt-mixed.c:67: GPGME: End of file
FAIL: t-encrypt-mixed
t-eventloop.c:205: GPGME: End of file
FAIL: t-eventloop
t-thread1.c:124: GPGME: Decryption failed
FAIL: t-thread1
PASS: t-thread-keylist
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:100: Unexpected fingerprint: 2D727CC768697734
FAIL: t-thread-keylist-verify
stopping gpg-agent
PASS: final.test
======================================
17 of 26 tests failed
Please report to http://bugs.gnupg.org
======================================
*** Error code 1

Installed packages:

libgpg-error-1.27
libassuan-2.4.3
libgcrypt-1.7.6
libksba-1.3.5
pinentry-1.0.0
libusb1-1.0.20
npth-1.3
gnupg21-2.1.18
python27-2.7.13

Btw, I had to add GPG=/usr/pkg/bin/gpg2 to the make command line so that gpg2 was used.

wiz created this task.Apr 5 2017, 11:15 AM
wiz created this object in space S1 Public.
wiz updated the task description. (Show Details)
wiz added a comment.Apr 5 2017, 11:20 AM

Oh, to make it clear - I was testing the pkgsrc version with the additional patches used by pkgsrc, see http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/security/gpgme/patches/

Testing it without patches does not work because:

get-env.c:57:2: error: #error Use of getenv_r not implemented.
 #error Use of getenv_r not implemented.
  ^
`
justus triaged this task as Normal priority.Jun 8 2017, 2:59 PM
marcus added a subscriber: marcus.Aug 10 2017, 5:31 PM

Well, we need more information to proceed on this. Maybe run with GPGME_DEBUG=9 to see why it fails.

wiz added a comment.Aug 29 2017, 2:30 PM

Sure. Here's the stdout and stderr for gpgme-1.9 with GPGME_DEBUG=9 and

libgpg-error-1.27
libassuan-2.4.3
libgcrypt-1.8.0
libksba-1.3.5
pinentry-1.0.0
libusb1-1.0.20
npth-1.5
gnupg2-2.2.0

gniibe claimed this task.
gniibe changed the task status from Open to Testing.
gniibe added projects: gnupg (gpg23), gpgagent.

I believe this is due to the bug of gpg-agent. So, I put this report as a sub task under T3276: the calibrate_get_time() function depends on a system that has a non-tickless kernel.

wiz added a comment.Nov 7 2017, 1:05 PM

I built gnupg 2.2.1 with the patch from D450, but that didn't help.
I even got an additional error:

dead lock detected
[1] Segmentation fault (core dumped) GNUPGHOME=/scrat...
FAIL: t-thread-keylist-verify

I added an #error into protect.c line 109 to make sure that the new block is compiled; that #error triggered.

There are multiple problems. I fixed one Makefile portability issue today.

werner added a subscriber: werner.Nov 21 2017, 10:14 AM
In T3056#95172, @wiz wrote:

Oh, to make it clear - I was testing the pkgsrc version with the additional patches used by pkgsrc, see http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/security/gpgme/patches/

Testing it without patches does not work because:

get-env.c:57:2: error: #error Use of getenv_r not implemented.
 #error Use of getenv_r not implemented.

That is the first time I have seen this. The cause for this is that configure didn't found that getenv is thread-safe and but an getenv_r exists. However gpgme did not make use of getenv_r - we need to fix that.

Other problems are fixed. Please test. It works for me on NetBSD 7.0.2.

wiz added a comment.Jan 29 2018, 10:37 AM

Using BSD make on git head of gpgme, I see

Making all in gpg
echo no-force-v3-sigs > ./gpg.conf
echo pinentry-program /archive/foreign/gpgme/tests/gpg/pinentry > ./gpg-agent.conf
make[3]: don't know how to make ./private-keys-v1.d/gpg-sample.stamp. Stop

With GNU make this error does not appear.

However, the build later fails with:

gmake[3]: Entering directory '/disk/6/archive/foreign/gpgme/lang/qt'
Making all in src
gmake[4]: Entering directory '/disk/6/archive/foreign/gpgme/lang/qt/src'
`test -f 'abstractimportjob.h' || echo './'`abstractimportjob.h -o abstractimportjob.moc
abstractimportjob.h: not found
Makefile:1164: recipe for target 'abstractimportjob.moc' failed
gmake[4]: *** [abstractimportjob.moc] Error 127
gmake[4]: Leaving directory '/disk/6/archive/foreign/gpgme/lang/qt/src'
Makefile:457: recipe for target 'all-recursive' failed
gmake[3]: *** [all-recursive] Error 1

I don't see how to disable the qt bindings from the configure script.

gniibe added a comment.EditedJan 29 2018, 11:04 AM

For the latter, I think it requires path to moc, which may be like /usr/pkg/qt5. Please add it to your PATH. Then, retry from configure

For BSD Make issue, please try:

wiz added a comment.EditedJan 29 2018, 11:59 AM

For qt: adding /usr/pkg/qt5/bin to the path makes the build succeed. I think you should take a look at the build rules though, since it seems that it wants to execute the header file if "moc" is not found.

When the build succeeded, I tried gmake check and most tests worked, except for:

********* Start testing of TestVarious *********
Config: Using QtTest library 5.10.0, Qt 5.10.0 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 5.5.0)
PASS   : TestVarious::initTestCase()
PASS   : TestVarious::testDN()
PASS   : TestVarious::testKeyFromFile()
FAIL!  : TestVarious::testQuickUid() 'key.numUserIDs() == 4' returned FALSE. ()
   Loc: [t-various.cpp(128)]
PASS   : TestVarious::testVersion()
PASS   : TestVarious::cleanupTestCase()
Totals: 5 passed, 1 failed, 0 skipped, 0 blacklisted, 1105ms
********* Finished testing of TestVarious *********
FAIL: t-various

As for the BSD make patch, that is not sufficient:

Making all in gpg
make[3]: don't know how to make ./private-keys-v1.d/gpg-sample.stamp. Stop

And one other thing: I ran autoreconf -fiv like I usually do, and that made git diff quite big -- lots of autogenerated files are checked into the repository which probably should not be there:

INSTALL
build-aux/compile
build-aux/config.guess
build-aux/config.sub
build-aux/depcomp
build-aux/install-sh
build-aux/ltmain.sh
build-aux/missing
build-aux/mkinstalldirs
build-aux/texinfo.tex
m4/libtool.m4
m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
m4/lt~obsolete.m4

Thanks for your additional suggestion. I pushed the change.

For autotool thing, in GnuPG project, we put some (like INSTALL, libtool files, config.*, etc.) into repo to avoid synchronizing those tools among developers, so, it's the maintainer's job to update those files.
For developers using git, it is expected to run ./autogen.sh, instead of autoreconf.
When done wrongly, you can reset by git reset --hard.

wiz added a comment.Feb 2 2018, 4:03 PM

I'm confused. I've just now retested, and I get further with BSD make (there is another problem when importing the keys into the test keyring, where it the error is ignored with GNU make but the build fails with BSD make) but that is not what I want to focus on.

I've moved gpgme-git version to a sandbox and installed just the dependencies, and tried running the tests again, and see 18 failures (with GNU make).

PASS: t-encrypt-sym
t-encrypt-sign.c:119: GPGME: End of file
FAIL: t-encrypt-sign
t-sign.c:127: GPGME: Unusable secret key
FAIL: t-sign
t-signers.c:124: GPGME: End of file
FAIL: t-signers
t-decrypt.c:69: GPGME: No secret key
FAIL: t-decrypt
t-verify.c:112: Unexpected signature summary: want=0x0 have=0x80
FAIL: t-verify
t-decrypt-verify.c:126: GPGME: No secret key
FAIL: t-decrypt-verify
t-sig-notation.c:181: GPGME: Unusable secret key
FAIL: t-sig-notation
Begin Result:
End Result.
t-export.c:74: GPGME: End of file
FAIL: t-export
PASS: t-import
PASS: t-trustlist
t-edit.c:140: GPGME: End of file
FAIL: t-edit
Primary key `Alfa Test' has unexpected key ID: AF82244F9CD9FD55
FAIL: t-keylist
Less keys returned than expected
FAIL: t-keylist-sig
PASS: t-wait
t-encrypt-large.c:127: GPGME: End of file
FAIL: t-encrypt-large
t-file-name.c:70: GPGME: End of file
FAIL: t-file-name
...
PASS: t-gpgconf
t-encrypt-mixed.c:68: GPGME: End of file
FAIL: t-encrypt-mixed
t-eventloop.c:205: GPGME: End of file
FAIL: t-eventloop
t-thread1.c:124: GPGME: No secret key
FAIL: t-thread1
PASS: t-thread-keylist
t-thread-keylist-verify.c:101: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:101: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:101: Unexpected fingerprint: 2D727CC768697734
t-thread-keylist-verify.c:101: Unexpected fingerprint: 2D727CC768697734
FAIL: t-thread-keylist-verify
stopping gpg-agent
PASS: final.test
======================================
18 of 26 tests failed
Please report to http://bugs.gnupg.org
======================================

I checked the process list, and there were a couple scdaemons and gpg2 running, so I killed them and tried again, but still have 17 errors (t-sig-notation worked this time).
Afterwards there was an scdaemon process still running.

I don't understand what changed to the successful tests last time; except that they were not in the sandbox, but the tests fail outside as well.

For other failures, I guess that you are connecting your card, aren't you?
Last year, I introduced a change for key selection to prefer existing card key. That may affect tests. Well, tests should have configure not to try to access card.

wiz added a comment.Feb 13 2018, 10:24 AM

No, I don't have a smartcard. Perhaps it misdetects one?

OK. Then, it may be some bashi-ism in Makefile. I'll investigate with no bash installed.

I located the problem. It's Makefile portability issue and it is fixed in: rMb5ec21b9baf0: tests: Makefile portability., rMba6e610baa13: tests: More Makefile portability., and rM3224d7f0ea83: tests: Fix previous commit
It was not your final invocation of "make check" (GNU or BSD), but the one before ("make all" by BSD make) which imported keys for tests.
The "export" directive doesn't work on BSD.

I believe that all BSD Makefile issues has been fixed (except python-tar-gz distribution thing for maintainer).
Please test again.

wiz added a comment.Feb 15 2018, 10:57 AM

Thank you very much! This is working quite well now.

There is only one test failure remaining for me:

********* Start testing of TestVarious *********
Config: Using QtTest library 5.10.0, Qt 5.10.0 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 5.5.0)
PASS   : TestVarious::initTestCase()
PASS   : TestVarious::testDN()
PASS   : TestVarious::testKeyFromFile()
FAIL!  : TestVarious::testQuickUid() 'key.numUserIDs() == 4' returned FALSE. ()
   Loc: [../../../../lang/qt/tests/t-various.cpp(128)]
PASS   : TestVarious::testVersion()
PASS   : TestVarious::cleanupTestCase()
Totals: 5 passed, 1 failed, 0 skipped, 0 blacklisted, 1093ms
********* Finished testing of TestVarious *********
FAIL: t-various

(automake should flag non-portable Makefile features - after all it is there to avoid gmake features)

The error of testQuickUID is strange. In the test, it adds a UID and checks number of UIDs (3 + 1 = 4).
It is not reproducible for me (Debian with Qt 5.9.2, NetBSD 7.0.2 with Qt 5.5.1), gnupg 2.2.x from the repo.

What are we going to do with this report? The last comment is 6 months old; can we change from testing to resolved or do we need to wait for a gpgme release?

werner moved this task from Backlog to For next release on the gpgme board.Aug 24 2018, 5:57 PM

I think it's good to close this as "resolved", since many fixes have been done, and I don't have remaining issue.
@wiz Please open another ticket for your next try.

gniibe closed this task as Resolved.Aug 28 2018, 4:20 AM