@werner what size of each additionally allocated secure memory area would you recommend? Is this something, that is better to set or leave up to the gpg-agent to decide? Will this additional memory be freed when not needed anymore or will it stay allocated until the process dies? I guess, the documentation could be expanded to answer this.

Thanks for the follow-up Werner.

Then better do not use a curses pinentry. It can't guarantee that another process changes the tty properties. For security reasons it is better to run the pinentry in a different window (ie. a GUI based pinentry).

If I understand correctly, the problem stems from the -module flag added to the LDFLAGS in commit dc2211179. It's that flag that instruct libtool to create a bundle (.so file) instead of a dynamically linked shared library (.dylib file). But that flag is needed to force automake to accept that the library is to be named scute instead of libscute (without that flag automake errors out, complaining that scute.la is not a standard libtool library name).

Given that 1.5 already had that problem, I would suggest to ignore that bug for the 1.6 release. We can work on that later.

For reference our downstream tracker of this is https://bugs.gentoo.org/683254 including patches

Added a fix to GnuPG, too (master and stable 2.2).

I keep this ticket open, since it is also problem for other packages.

Your problem is apparently not an issue of upstream development of GnuPG; It is your setup script (agent.sh?) which specifies /dev/shm/SOMETHING.
Standard GnuPG never does that. We have no idea about use of /dev/shm/SOMETHING.

Please put

I ran gpgconf --kill gpg-agent and then the suggested command for i in {1..10}; do gpg -v --no-tty --verbose -o - encrypted.gpg 2>mylog.$i > /dev/null & done. (I was already running with --verbose, does -v add something else?)

In the interests of completeness I also tried it on a much larger file (1GB) which was both signed and encrypted. I also set the decryption to show the session key just to confirm it was decrypting since the plaintext was being sent to /dev/null.

I am unable to replicate this on OS X 10.9 Mavericks.

Running with -v would really be helpful.

I made a large file for testing, but it doesn't matter. There's an arbitrary parallel limit where gpg will crash.

We need a way to replicate your problem, a few questions first:

Checked - it builds fine now. Thanks!

I push my change to master.
Please test.

Thanks for testing master.
No, it's not typo, in my opinion.
The line was added as if it's LOCAL_PEERUID, but there is no such a thing in XNU, but there is LOCAL_PEERUUID which is for UUID.

It will be in the next release (2.4.4).
Thanks for reporting.

The description of this bug report is not correct.
_POSIX_C_SOURCE should *not* be defined to use INADDR_LOOPBACK for the system.

It looks like this was on my side. I can't reproduce it anymore; in other words dirmngr survives changes to DNS servers now.

That's it. I can reproduce this on Debian.

debug dns

log-file whateveryouwant

You're right, stat() works correctly. I created a small tool that implements the same logic. For some reason dirmngr is still not able to find the DNS server after suspend/resume in combination with changed locations. I still get "no route to host" errors.

According to POSIX stat(2) follows a symlink and thus /etc/resolv.conf is the right name to use. (To stat /etc/resolv.conf itself lstat(2) would need to be used. ). I just checked the macOS man page and it says nothing to the contrary.

I've found that I can get the test to succeed if I drop --disable-scdaemon from my configure flags. I'm far from qualified to diagnose this, but I suspect that the tests have a bug in which they still try to test the scdaemon despite the presence of --disable-scdaemon in the configure flags.

Sure it won't apply because it is part of 2.1.22. ;-)

No way to test on El Capitain anymore. It works on Sierra.

Given that this is just a warning, we should not consider it a bug.

Can confirm here too. Applying that on top of 2.1.21 works perfectly.

Yes that fixes it!

I put another bug in 2.1.21. Please try: rGa8dd96826f84: g10: Suppress error for card availability check.

Unsure whether to bump this or report it as a fresh bug, but the testing-scdaemon-inside-a-sandbox-on-macos issue has returned in GnuPG 2.1.21.

This is likely a duplicate or something really closely related to T3080.

https://developer.apple.com/library/content/documentation/Networking/Conceptual/Open_Directory/openDirectoryConcepts/openDirectoryConcepts.html

Or provide an option to disable LDAP: T2908: dirmngr can't be build w/o LDAP

Applied as ebe12be034f0.

While I can't reproduce this problem myself, I think I found an issue of gpg-agent passphrase caching.
Double free may happen when multiple threads enter agent_put_cache, for example.

In 2.1.19, gpg-agent uses getpeerucred for macOS. I changed it (since it seemed not working). In 2.1.20, gpg-agent now uses getsockopt with LOCAL_PEERPID.
It seems for me that the crash occurs by ucred_free. If this is the case, 2.1.20 fixes this issue.

4ce4f2f683a17be3ddb93729f3f25014a97934ad allows make check to complete without
the other workaround. So it works as advertised! Thanks, Niibe and Justus.

gnupg fixed in dd60e868d2bf649a33dc96e207ffd3b8ae4d35af.

ntbtls fixed in e582e91e47a164816ac074b9078dbed8537601dc.

libgcrypt fixed in 654024081cfa103c87bb163b117ea3568171d408.

libksba fixed in 561d03a008150c201ece22b29c97b24a1f6bf590.

libassuan fixed in b26b73d04bff10852382113ae361ea5726661510.

libgpg-error fixed in 5e51b642f747547c737a7abbc37e65b0f630d188.

Sorry, I couldn't find any possible bug for PC/SC access in scdaemon. It looks
like scdaemon crashes when it tries to access card by PC/SC, and it seems that
it crashes there (I mean, in PC/SC).
I believe that this scdaemon's crash is something which is difficult to avoid in
an application.

Anyway, I fixed the issue itself by handling errors of gpg-agent for scdaemon:
https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=4ce4f2f683a17be3ddb93729f3f25014a97934ad

I think that scdaemon in 2.1.18 would also crash in sandbox environment.
In 2.1.19, I modified ssh-agent emulation code to support multiple tokens.
This change assumes scdaemon returns ENODEV return code and behaves badly, if
scdaemon crashes.
In 2.1.18, the code was somewhat robust and scdaemon crash didn't cause failure.
I am currently looking into the reason why scdaemon crashes.

Thanks to you and thanks to Niibe in advance :)

Glad to hear that. Niibe will have a closer look at the issue tomorrow.

And it has now passed CI:
https://github.com/Homebrew/homebrew-versions/pull/1536
https://bot.brew.sh/job/Homebrew%20Versions%20Pull%20Requests/1824/

So I've merged the PR.

The patch works!

A possible workaround for you is to disable the use of scdaemon in the tests.
Currently, it is not needed anyway. I'll attach a patch to do that.

D412: 966_issue2980-workaround.patch

This is interesting indeed. Might be related to a recent change to support
multiple smart cards.

Here's the referenced crash file:
https://gist.github.com/ilovezfs/ebfccf2515fc7d7952edcc4c13ff8013

Here's what's happening in the system log when the failed test runs:

Mar  2 03:06:53 iMac-TMP com.apple.xpc.launchd[1] (com.apple.auditd[39537])
<Warning>: Endpoint has been activated through legacy launch(3) APIs. Please
switch to XPC or bootstrap_check_in(): com.apple.auditd
Mar  2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Notice>:
SecTaskLoadEntitlements failed error=22
Mar  2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed
PCSC.framework client without com.apple.security.smartcard entitlement
Mar  2 03:06:53 iMac-TMP joe[39540] <Warning>: audit warning: soft /var/audit
Mar  2 03:06:53 iMac-TMP joe[39541] <Warning>: audit warning: allsoft 
Mar  2 03:06:54 iMac-TMP joe[39544] <Warning>: audit warning: closefile
/var/audit/20170302110453.20170302110653
Mar  2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Notice>:
SecTaskLoadEntitlements failed error=22
Mar  2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed
PCSC.framework client without com.apple.security.smartcard entitlement
Mar  2 03:06:54 iMac-TMP com.apple.xpc.launchd[1] (com.apple.ReportCrash[39548])
<Warning>: Endpoint has been activated through legacy launch(3) APIs. Please
switch to XPC or bootstrap_check_in(): com.apple.ReportCrash
Mar  2 03:06:54 iMac-TMP ReportCrash[39548] <Notice>: Saved crash report for
scdaemon[39535] version 0 to
/Users/joe/Library/Logs/DiagnosticReports/scdaemon_2017-03-02-030654_iMac-TMP.crash

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

It's the system default. There's no other version of ssh that gets installed.
Our own ssh formula is homebrew/dupes/openssh and is explicitly barred from
being used as a dependency by anything else, as is anything else in homebrew/dupes.

10.12.3
robotunicorn ~ # ssh -V
OpenSSH_7.3p1, LibreSSL 2.4.1

10.11.6
iMac-TMP:~ joe$ ssh -V
OpenSSH_6.9p1, LibreSSL 2.1.8

yosemitevm ~ # ssh -V
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011

Regarding the sandbox, here's where it's implemented:
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/sandbox.rb

It's invoked as

/usr/bin/sandbox-exec -f /tmp/homebrew20170302-24230-1xmlw7l.sb nice
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -W0 -I
/usr/local/Homebrew/Library/Homebrew --
/usr/local/Homebrew/Library/Homebrew/build.rb
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-versions/gnupg21.rb

The contents of the .sb file are as follows:

iMac-TMP:~ joe$ cat /tmp/homebrew20170302-24230-1xmlw7l.sb
(version 1)
(debug deny) ; log all denied operations to /var/log/system.log
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/private/var/tmp"))
(allow file-write* (regex #"^/private/var/folders/[^/]+/[^/]+/[C,T]/"))
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/Users/joe/Library/Caches/Homebrew"))
(allow file-write* (subpath "/Users/joe/Library/Logs/Homebrew/gnupg21"))
(allow file-write* (subpath "/Users/joe/Library/Developer"))
(allow file-write* (subpath "/usr/local/Cellar/gnupg21"))
(allow file-write* (subpath "/usr/local/etc"))
(allow file-write* (subpath "/usr/local/var"))
(allow file-write*
    (literal "/dev/ptmx")
    (literal "/dev/dtracehelper")
    (literal "/dev/null")
    (literal "/dev/zero")
    (regex #"^/dev/fd/[0-9]+$")
    (regex #"^/dev/ttys?[0-9]*$")
    )
(deny file-write*) ; deny non-whitelist file write operations
(allow process-exec
    (literal "/bin/ps")
    (with no-sandbox)
    ) ; allow certain processes running without sandbox
(allow default) ; allow everything else

The environment variables themselves are not different between sandboxed and
non-sandboxed builds.

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

(I'm mildly annoyed that I have to ask again. You are not forthcoming with
information about your environment. macOS is somewhat alien for us, and if you
want help with tracking down the issue, you need to give us more information.
Note that we really do make an effort, and we have a macOS build slave that does
not see these issues:

https://jenkins.gnupg.org/job/gnupg/XTARGET=native,label=macos/

(though we get our build dependencies from pkgsrc, and you get it from your
packages I guess, so there are bound to be differences).)

From your latest log I see that the version of ssh used supports ed25519, so
this must be version newer than 6.5.

I just committed a patch that dumps the ssh version and the path to the binary
when executing the test:

https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commitdiff;h=74cb3b230c1f99afc5fd09bccc24186a63b154b0

Here's the successful log with --no-sandbox:
https://gist.githubusercontent.com/ilovezfs/a886421569e625d0b7051cf8e9bfea53/raw/77aec417f54ed3d996eb340f826de9d7569088f5/gistfile1.txt

This is not about socket directories, and SSH_AUTH_SOCK is set properly (as
demonstrated by the fact that dsa and rsa works).