Yeah, unfortunately this is still happening even with my attempt to fix it with
deparallelization, so that's not the issue.

macOS 10.9 build bot has failed in the same place again.

Here's the relevant snippet:

PASS: tofu.scm
Checking gpgtar without encryption 
Checking gpgtar without encryption with nicer actions 
Checking gpgtar with asymmetric encryption 
Checking gpgtar with asymmetric encryption and signature 
Checking gpgtar with signature 
((/private/tmp/gnupg21-20160725-43964-l18ixl/gnupg-2.1.14/tools/gpgtar --gpg
/private/tmp/gnupg21-20160725-43964-l18ixl/gnupg-2.1.14/g10/gpg --gpg-args
--no-permission-warning --always-trust --tar-args --directory=. --decrypt
/tmp/gpgscm-PgAlmV/archive) failed: gpgtar: gpg: [don't know]: invalid packet
(ctb=2d)
gpgtar: gpg: [don't know]: invalid packet (ctb=2a)
gpgtar: error running
'/private/tmp/gnupg21-20160725-43964-l18ixl/gnupg-2.1.14/g10/gpg': exit status 2
) 
FAIL: gpgtar.scm
Importing public key. 
Checking that the most recent, valid signing subkey is used by default 
    > 8BC90111 3E880CFF F5F77B83 45117079 1EA97479 <  
Checking that we can select a specific signing key 
    > 8BC90111 F5F77B83 1EA97479 <  
PASS: use-exact-key.scm
Importing public key.

I have now tested HEAD at revision 4ba11251aff578394000bf480f47160f0879c763 and
2.1.13 (including this patch
https://raw.githubusercontent.com/Homebrew/formula-patches/7b2211b/gnupg21/spawned_child_8f79c31b.diff)

The results are

1. The "FAIL: gpgtar.scm" does *not* affect the patched 2.1.13, so it appears to

be a regression.

2. The "FAIL: gpgtar.scm" is not fixed in HEAD at

4ba11251aff578394000bf480f47160f0879c763.

3. There is a further regression in HEAD at

4ba11251aff578394000bf480f47160f0879c763, which seems not to be found in 2.1.14:

PASS: default-key.scm
Checking key export 
    > D74C5F22 C40FDECF ECABF51D <  
PASS: export.scm
Importing ssh keys... 
    > dsa key not added 
FAIL: ssh.scm
Checking passphrase cache (issue2015)... 
PASS: T2015.scm
Checking import statistics (issue2346)... 
PASS: T2346.scm

Thanks for letting us know. Unfortunately, we do not test on MacOS yet, but we are working
on that.

I have neither experience with debugging on MacOS, nor do I have access to such a machine.
I'm afraid you are on your own for now.

The ssh test is new, so we need to figure out why it does not work. Please do

  make -C tests/openpgp check TESTS="setup.scm ssh.scm" verbose=2

This lets us see what ssh-add prints to stderr. It might be related to the version of
OpenSSH shipped with the OS.

Hi Justus,

Thanks for your response. In further testing, I was able to trigger the "FAIL:
gpgtar.scm" during a make check for 2.1.13 (actually "FAIL: gpgtar.test" for
2.1.13 since it's pre-tiny-scheme). In particular, it's vanilla 2.1.13 + your
fix in 8f79c31b. So I think what may be going on is that 8f79c31b didn't
actually fully resolve that problem after all since I've now seen it occur, with
that commit included, in 2.1.13, and in 2.1.14, and in HEAD.

Tbere were two cases where a more specific error was emitted:

In one run, I saw this:

((/private/tmp/gnupg21-20160725-43964-l18ixl/gnupg-2.1.14/tools/gpgtar --gpg
/private/tmp/gnupg21-20160725-43964-l18ixl/gnupg-2.1.14/g10/gpg --gpg-args
--no-permission-warning --always-trust --tar-args --directory=. --decrypt
/tmp/gpgscm-PgAlmV/archive) failed: gpgtar: gpg: [don't know]: invalid packet
(ctb=2d)
gpgtar: gpg: [don't know]: invalid packet (ctb=2a)
gpgtar: error running
'/private/tmp/gnupg21-20160725-43964-l18ixl/gnupg-2.1.14/g10/gpg': exit status 2
) 
FAIL: gpgtar.scm

and in another run I saw this

Checking gpgtar with signature 
((/private/tmp/gnupg21-20160726-74591-maikty/gnupg-2.1.14/tools/gpgtar --gpg
/private/tmp/gnupg21-20160726-74591-maikty/gnupg-2.1.14/g10/gpg --gpg-args
--no-permission-warning --always-trust --tar-args --directory=. --decrypt
/tmp/gpgscm-0U4bUB/archive) failed: gpgtar: gpg: Fatal: zlib inflate problem:
invalid block type
gpgtar: error running
'/private/tmp/gnupg21-20160726-74591-maikty/gnupg-2.1.14/g10/gpg': exit status 2
) 
FAIL: gpgtar.scm

It's also worth noting that I've only been able to trigger the problem on
Jenkins during CI, not locally, so I don't know if the lack of TTY is relevant
or something like that.

I will do the ssh check you requested.

Can you please tell us what version of ssh you are using (ssh -V)?

We now have a macOS box, and are building our software on it using Jenkins.

On that box, I also see the gpgtar test failing in about 14% of all runs. There
is something to be learned here.

The ssh.scm failure is still happening intermittently with 2.1.16

https://bot.brew.sh/job/Homebrew%20Versions%20Pull%20Requests/1733/version=yosemite/console

$ ssh -V
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011

Let's use T2425 for the tar failure, and T2847 for the ssh failure. The
log you posted here shows exactly the same problem as in T2847.

Do you also see tar failing?

You can use

make -Ctests/openpgp check XTESTS="gpgtar.scm gpgtar.scm gpgtar.scm gpgtar.scm
gpgtar.scm"

to run the same test over and over again. That is how I measured how often we
see the failure. We updated our box since, and I haven't tried it again yet.

Fixed in 34fa2d79a07a079be472c3ff486debfdac8c6070.