This is interesting indeed. Might be related to a recent change to support
multiple smart cards.

Here's the referenced crash file:
https://gist.github.com/ilovezfs/ebfccf2515fc7d7952edcc4c13ff8013

Here's what's happening in the system log when the failed test runs:

Mar  2 03:06:53 iMac-TMP com.apple.xpc.launchd[1] (com.apple.auditd[39537])
<Warning>: Endpoint has been activated through legacy launch(3) APIs. Please
switch to XPC or bootstrap_check_in(): com.apple.auditd
Mar  2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Notice>:
SecTaskLoadEntitlements failed error=22
Mar  2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed
PCSC.framework client without com.apple.security.smartcard entitlement
Mar  2 03:06:53 iMac-TMP joe[39540] <Warning>: audit warning: soft /var/audit
Mar  2 03:06:53 iMac-TMP joe[39541] <Warning>: audit warning: allsoft 
Mar  2 03:06:54 iMac-TMP joe[39544] <Warning>: audit warning: closefile
/var/audit/20170302110453.20170302110653
Mar  2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Notice>:
SecTaskLoadEntitlements failed error=22
Mar  2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed
PCSC.framework client without com.apple.security.smartcard entitlement
Mar  2 03:06:54 iMac-TMP com.apple.xpc.launchd[1] (com.apple.ReportCrash[39548])
<Warning>: Endpoint has been activated through legacy launch(3) APIs. Please
switch to XPC or bootstrap_check_in(): com.apple.ReportCrash
Mar  2 03:06:54 iMac-TMP ReportCrash[39548] <Notice>: Saved crash report for
scdaemon[39535] version 0 to
/Users/joe/Library/Logs/DiagnosticReports/scdaemon_2017-03-02-030654_iMac-TMP.crash

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

It's the system default. There's no other version of ssh that gets installed.
Our own ssh formula is homebrew/dupes/openssh and is explicitly barred from
being used as a dependency by anything else, as is anything else in homebrew/dupes.

10.12.3
robotunicorn ~ # ssh -V
OpenSSH_7.3p1, LibreSSL 2.4.1

10.11.6
iMac-TMP:~ joe$ ssh -V
OpenSSH_6.9p1, LibreSSL 2.1.8

yosemitevm ~ # ssh -V
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011

Regarding the sandbox, here's where it's implemented:
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/sandbox.rb

It's invoked as

/usr/bin/sandbox-exec -f /tmp/homebrew20170302-24230-1xmlw7l.sb nice
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -W0 -I
/usr/local/Homebrew/Library/Homebrew --
/usr/local/Homebrew/Library/Homebrew/build.rb
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-versions/gnupg21.rb

The contents of the .sb file are as follows:

iMac-TMP:~ joe$ cat /tmp/homebrew20170302-24230-1xmlw7l.sb
(version 1)
(debug deny) ; log all denied operations to /var/log/system.log
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/private/var/tmp"))
(allow file-write* (regex #"^/private/var/folders/[^/]+/[^/]+/[C,T]/"))
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/Users/joe/Library/Caches/Homebrew"))
(allow file-write* (subpath "/Users/joe/Library/Logs/Homebrew/gnupg21"))
(allow file-write* (subpath "/Users/joe/Library/Developer"))
(allow file-write* (subpath "/usr/local/Cellar/gnupg21"))
(allow file-write* (subpath "/usr/local/etc"))
(allow file-write* (subpath "/usr/local/var"))
(allow file-write*
    (literal "/dev/ptmx")
    (literal "/dev/dtracehelper")
    (literal "/dev/null")
    (literal "/dev/zero")
    (regex #"^/dev/fd/[0-9]+$")
    (regex #"^/dev/ttys?[0-9]*$")
    )
(deny file-write*) ; deny non-whitelist file write operations
(allow process-exec
    (literal "/bin/ps")
    (with no-sandbox)
    ) ; allow certain processes running without sandbox
(allow default) ; allow everything else

The environment variables themselves are not different between sandboxed and
non-sandboxed builds.

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

(I'm mildly annoyed that I have to ask again. You are not forthcoming with
information about your environment. macOS is somewhat alien for us, and if you
want help with tracking down the issue, you need to give us more information.
Note that we really do make an effort, and we have a macOS build slave that does
not see these issues:

https://jenkins.gnupg.org/job/gnupg/XTARGET=native,label=macos/

(though we get our build dependencies from pkgsrc, and you get it from your
packages I guess, so there are bound to be differences).)

From your latest log I see that the version of ssh used supports ed25519, so
this must be version newer than 6.5.

I just committed a patch that dumps the ssh version and the path to the binary
when executing the test:

https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commitdiff;h=74cb3b230c1f99afc5fd09bccc24186a63b154b0

Here's the successful log with --no-sandbox:
https://gist.githubusercontent.com/ilovezfs/a886421569e625d0b7051cf8e9bfea53/raw/77aec417f54ed3d996eb340f826de9d7569088f5/gistfile1.txt

This is not about socket directories, and SSH_AUTH_SOCK is set properly (as
demonstrated by the fact that dsa and rsa works).

mkdir /run/user/YOURUID

/run is not a thing on macOS. /var/run is but requires root access. The default
socket location on macOS is

iMac-TMP:~ joe$ ls -al /tmp/com.apple.launchd.OWt2QNcw7V
total 0
drwx------   2 joe   wheel   102 Mar  1 21:13 .
drwxrwxrwt  76 root  wheel  2686 Mar  2 00:44 ..
srw-rw-rw-   1 joe   admin     0 Mar  1 21:13 Listeners

One such directory and socket are created for each user when using ssh.

Note that I had tried hacking in /usr/local/var/run in place of your /run, and
creating the directory, etc., which the test did proceed to use but it didn't
resolve the problem. (I had done that before reporting here.)

Yup! And remember, that was only a <= 10.10 issue, whereas the new "fun" is
across the board, 10.10, 10.11, and 10.12 (probably <= 10.9 too but we don't
test those).

Oh cool, so that patch was effective :)

Your T2947 says otherwise. So does T2847. Can you please clarify that?

Sure. I'm referring to 2.1.18 with your subsequent fixes. See
https://github.com/Homebrew/homebrew-versions/commit/8243792932da5b7746bf229d4b63abd7592b21b3
which has the relevant patches (thank you) on the left side of the diff.

2.1.18 did not have this problem, and the tests completed successfully within
the sandboxed Homebrew build environment.

Your T2947 says otherwise. So does T2847. Can you please clarify that?

This is the interesting part of the log:

Importing ssh keys...

> dsa Executing: '/usr/bin/ssh-add' '-'

Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
rsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
ecdsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
error fetching identities for protocol 1: agent refused operation
error fetching identities for protocol 2: agent refused operation

So, what happens for 'dsa', 'rsa', 'ecdsa' is that first the key is added, then
with ssh-add -l we check that it is in fact added. This works for dsa and rsa,
but fails for ecdsa.

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

This happens on at least PPC Mac OS X 10.4.11, Tiger. Compiler is by default Apple's
version of GCC 4.2. The error is reported as this:

libtool: compile: /opt/local/bin/gcc-apple-4.2 -DHAVE_CONFIG_H -I. -I..
-I.. -I/opt/local/include -I/opt/local/include -pipe -Os -arch ppc -Wall
-Wcast-align -Wshadow -Wstrict-prototypes -Wpointer-arith -MT
libassuan_la-assuan-socket.lo -MD -MP -MF .deps/libassuan_la-assuan-
socket.Tpo -c assuan-socket.c -fno-common -DPIC -o .libs/libassuan_la-
assuan-socket.o
assuan-socket.c: In function 'socks5_connect':
assuan-socket.c:732: error: 'INADDR_LOOPBACK' undeclared (first use in
this function)
assuan-socket.c:732: error: (Each undeclared identifier is reported only
once
assuan-socket.c:732: error: for each function it appears in.)
make[3]: * [libassuan_la-assuan-socket.lo] Error 1
make[3]: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3/src'
make[2]: * [all] Error 2
make[2]: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3/src'
make[1]: * [all-recursive] Error 1
make[1]: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3'
make: * [all] Error 2
make: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3'
Command failed: cd
"/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3"
&& /usr/bin/make -w all

Cause is that the declaration of 'INADDR_LOOPBACK' is hidden behind some guards. The
two external links document the situation, they also offer patches, either the attached
one or this addition for src/assuan-socket.c:

//fix missing define in MacOSX 10.4 Tiger
#ifndef INADDR_LOOPBACK

#define INADDR_LOOPBACK (u_int32_t)0x7f000001

#endif

In ten days I'll be at home again with my Apple PowerBook G4 and might have time to
check the situation in Mac OS X 10.5, Leopard.

D380: 963_src_assuan-socket_INADDR_LOOPBACK.patch

I addressed this for GPGME in 60273e8b2c11d42215a5707bc55e3e0d8f350e07 but
apparently forgot to mention that here.

I'll keep the bug open until I fixed this in all packages.

Addressed in 56aa85f88f6b35fb03a2dc1a95882d49a74290e3.

Here's the make check verbose=2 log for 2.1.18 on macOS 10.10.5 Yosemite:
https://gist.github.com/ilovezfs/d9de58955697858a1eb3c6d3a5e48bea

And ssh -V:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011

Released with 2.1.18

Fixed in 34fa2d79a07a079be472c3ff486debfdac8c6070.

FWIW: In GnuPG we have for example this in the configure script

*-*-hpux*)
    if test -z "$GCC" ; then
        CFLAGS="-Ae -D_HPUX_SOURCE $CFLAGS"
    fi

IF it makes things easier we may add a simlar case for macOS. But we need to do
this for many packages, I think.

We configure the build with -D_DARWIN_C_SOURCE=900000L on our macOs box. Not
sure if this is the proper thing to do, and/or if we should just always set that
when we build on Darwin in configure.

Thanks, applied!

Forgot to give you credit / mention this bug in the commit message. Apologies
for that.

FWIW, we do not see any problem with current Libgcrypt in our CUI system running
under Sierra.

FWIW, we are running build tests now on macOS Sierra w/o problems.

Also:

$ ssh -V
OpenSSH_7.2p2, LibreSSL 2.4.1

Thanks for the report.

I changed the title to reflect what I learned from the log.

Our test runs fine, here a recent the log:

http://jenkins.gnupg.org/job/gnupg/501/XTARGET=native,label=macos/consoleFull

I don't know how to compare the OS versions, but this is what I see:

$ uname -a
Darwin ... 16.0.0 Darwin Kernel Version 16.0.0: Mon Aug 29 17:56:20 PDT 2016;
root:xnu-3789.1.32~3/RELEASE_X86_64 x86_64
$ shasum /usr/bin/ssh-add
bdb1005292b0891edba78b3f1f00fe036c4e60f9 /usr/bin/ssh-add

Could you please arrange the tests to be called using 'make check verbose=2',
and post
the generated ssh.scm.log file? For reference, here is our log:

http://jenkins.gnupg.org/job/gnupg/XTARGET=native,label=macos/ws/obj/tests/openpgp/ssh-import.scm.log/*view*/

(Note that I just renamed the test to 'ssh-import.scm'.)

Ah I spoke too soon. Just got the ssh.scm:
https://bot.brew.sh/job/Homebrew%20Versions%20Pull%20Requests/1733/version=yosemite/console

Everything looks fine now that I removed all of the dependencies and started
from a blank slate. Sorry for the noise.

So far I'm not seeing the old "FAIL: gpgtar.scm" and "FAIL: ssh.scm"

Were those specifically fixed in some new commit(s), or am I just lucky so far?

I used your workaround and haven't been running into problems since. Unfortunately, I
don't currently have the time at hand to give it a thorough test run. If I do, I will
keep you updated.

Could you verify that the problem has been solved (in 2.1.14)?

I see. We use system() in the random test to re-execute itself. This involves
the shell and thus the problem. Need to uses fork/exec or CreateProcess calls
for that test. I guess this needs to wait until we have moved that to code to
libgpg-error as our portability layer.

This is still a problem on OS X 10.11.5. OS X's System Integrity Protection
"feature" is causing that test failure. If S.I.P is disabled there's no problem.

A similar-looking test failure happens in perl
(https://rt.perl.org/Public/Bug/Display.html?id=126706). Perhaps the diagnosis is
the same here.

I am not sure about the cause for this bug. However it might be fixed either be
2.1.3 (released a few days ago) or libgpg-error 1.23.

Workaround: Use
ggp --export-ownertrust >ot.txt
rm trustdb.gpg
gpg --import-ownertrust <ot.txt

No updates in 8 months, thus closing.

Closing, I assume it's the same bug of 2112, which was fixed.

Perhaps, this is same bug in T2112

AFAIK, this is a bug in oner version of the OSX toolchain. Others have shown
that it is possible to build the vanilla tarball on OSX. If your problem
persists, please search the gnupg-users mailing lits and ask there for help.

I have no idea myself. We do not use libdispacth direclty but it may be used
indirectkly be socket or threads code. Thus I would start to look at libnpth -
but it won't be easy to find such a problem.

Thanks for the advice! I totally cargo-culted those flags from some tutorial on the
Internet, so I'm not surprised they're out of whack.

I can hook up a debugger and investigate myself if you give me some idea of where to
look and what to look for.

Thanks for the report. However I am not rigged to replicate this thus we need
to wait for someone elese to look at this.

BTW, you should not use --write-env-file and --use-standard-socket because they
are the defaults in 2.1. --scdaemon-program is only a debug option. GnuPG
knows the full names of all its modules.

This 1st step debugging modification on configure

• configure 2015/06/03 14:56:54 1.1
• configure 2015/06/03 14:57:45
• 14873,14878 ****
• 14873,14879 ---- else HAVE_INTTYPES_H=0 fi

+ HAVE_INTTYPES_H=0

      if test $ac_cv_header_sys_types_h = yes; then

permit a complete make && make check

The patch is a work for problem somewhere in the PC/SC implementaion. I am also
not sure whether a pthread_cancel for a buggy PC/SC library is a good idea.
Terminating the process seems to be a better solution.

If gpgtools wants to apply this pacth, they might of course do so but I don't
want to apply it upstream in particular not to an older version (2.1 is current).

That seems to be a OS/X specific bug in its file systems. I have no idea how to
fix that. Maybe this should go into the FAQ. The obvious workaround is to copy
a pubring.gpg with one key to the MSDOS directory.

You were right, of course! Thanks for the tipp, which revealed it immediately.
Please close this ticket.

That is a warning to tell you that you called a regular Libgcrypt function
before gcry_check_version. It auto-initializes Libgcrypt as a workaround.

Trivial patch. Closing bug,.