Hum, there is something strange going on here:
% gpg --auto-key-locate wkd --locate-key root@leckerlecker
... no update...
% gpg --auto-key-locate wkd,local --locate-key root@leckerlecker
... no update...
% gpg --auto-key-locate clear,wkd,local --locate-key root@leckerlecker
... update!...
Thanks to you and thanks to Niibe in advance :)
Glad to hear that. Niibe will have a closer look at the issue tomorrow.
Fixed in 4735ab96aa5577d40ba7b3f72d863057198cc6a7.
And it has now passed CI:
https://github.com/Homebrew/homebrew-versions/pull/1536
https://bot.brew.sh/job/Homebrew%20Versions%20Pull%20Requests/1824/
So I've merged the PR.
The patch works!
I read the code and documentation for key-edit's revuid, and --quick-revoke-uid,
and the code makes no effort to ensure that one valid UID remains.
I read rfc4880 trying to find the basis for "some non-revoked User ID must
remain", but the only justification I could find is in section 11.1.,
Transferable Public Keys, that states that at least one UID must be included if
one wants to transfer keys.
So, do we actually want to enforce that or fix the documentation?
A possible workaround for you is to disable the use of scdaemon in the tests.
Currently, it is not needed anyway. I'll attach a patch to do that.
This is interesting indeed. Might be related to a recent change to support
multiple smart cards.
Here's the referenced crash file:
https://gist.github.com/ilovezfs/ebfccf2515fc7d7952edcc4c13ff8013
Here's what's happening in the system log when the failed test runs:
Mar 2 03:06:53 iMac-TMP com.apple.xpc.launchd[1] (com.apple.auditd[39537]) <Warning>: Endpoint has been activated through legacy launch(3) APIs. Please switch to XPC or bootstrap_check_in(): com.apple.auditd Mar 2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Notice>: SecTaskLoadEntitlements failed error=22 Mar 2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed PCSC.framework client without com.apple.security.smartcard entitlement Mar 2 03:06:53 iMac-TMP joe[39540] <Warning>: audit warning: soft /var/audit Mar 2 03:06:53 iMac-TMP joe[39541] <Warning>: audit warning: allsoft Mar 2 03:06:54 iMac-TMP joe[39544] <Warning>: audit warning: closefile /var/audit/20170302110453.20170302110653 Mar 2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Notice>: SecTaskLoadEntitlements failed error=22 Mar 2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed PCSC.framework client without com.apple.security.smartcard entitlement Mar 2 03:06:54 iMac-TMP com.apple.xpc.launchd[1] (com.apple.ReportCrash[39548]) <Warning>: Endpoint has been activated through legacy launch(3) APIs. Please switch to XPC or bootstrap_check_in(): com.apple.ReportCrash Mar 2 03:06:54 iMac-TMP ReportCrash[39548] <Notice>: Saved crash report for scdaemon[39535] version 0 to /Users/joe/Library/Logs/DiagnosticReports/scdaemon_2017-03-02-030654_iMac-TMP.crash
Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?
It's the system default. There's no other version of ssh that gets installed.
Our own ssh formula is homebrew/dupes/openssh and is explicitly barred from
being used as a dependency by anything else, as is anything else in homebrew/dupes.
10.12.3
robotunicorn ~ # ssh -V
OpenSSH_7.3p1, LibreSSL 2.4.1
10.11.6
iMac-TMP:~ joe$ ssh -V
OpenSSH_6.9p1, LibreSSL 2.1.8
yosemitevm ~ # ssh -V
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
Regarding the sandbox, here's where it's implemented:
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/sandbox.rb
It's invoked as
/usr/bin/sandbox-exec -f /tmp/homebrew20170302-24230-1xmlw7l.sb nice /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-versions/gnupg21.rb
The contents of the .sb file are as follows:
iMac-TMP:~ joe$ cat /tmp/homebrew20170302-24230-1xmlw7l.sb
(version 1)
(debug deny) ; log all denied operations to /var/log/system.log
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/private/var/tmp"))
(allow file-write* (regex #"^/private/var/folders/[^/]+/[^/]+/[C,T]/"))
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/Users/joe/Library/Caches/Homebrew"))
(allow file-write* (subpath "/Users/joe/Library/Logs/Homebrew/gnupg21"))
(allow file-write* (subpath "/Users/joe/Library/Developer"))
(allow file-write* (subpath "/usr/local/Cellar/gnupg21"))
(allow file-write* (subpath "/usr/local/etc"))
(allow file-write* (subpath "/usr/local/var"))
(allow file-write*
(literal "/dev/ptmx")
(literal "/dev/dtracehelper")
(literal "/dev/null")
(literal "/dev/zero")
(regex #"^/dev/fd/[0-9]+$")
(regex #"^/dev/ttys?[0-9]*$")
)
(deny file-write*) ; deny non-whitelist file write operations
(allow process-exec
(literal "/bin/ps")
(with no-sandbox)
) ; allow certain processes running without sandbox
(allow default) ; allow everything elseThe environment variables themselves are not different between sandboxed and
non-sandboxed builds.
From what I've seen there is no variation in getsockname, it just returns
whatever path is passed to bind. I don't understand the need for getsockname
tbh, because we are the ones that bind the socket in the first place.
(The only variation seems to be that the function is broken on Hurd...).
Fixed in 62d21a4ab4029b32ea129f1cf3a0e1f22e2fb7b0.
Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?
(I'm mildly annoyed that I have to ask again. You are not forthcoming with
information about your environment. macOS is somewhat alien for us, and if you
want help with tracking down the issue, you need to give us more information.
Note that we really do make an effort, and we have a macOS build slave that does
not see these issues:
https://jenkins.gnupg.org/job/gnupg/XTARGET=native,label=macos/
(though we get our build dependencies from pkgsrc, and you get it from your
packages I guess, so there are bound to be differences).)
From your latest log I see that the version of ssh used supports ed25519, so
this must be version newer than 6.5.
I just committed a patch that dumps the ssh version and the path to the binary
when executing the test:
Here's the successful log with --no-sandbox:
https://gist.githubusercontent.com/ilovezfs/a886421569e625d0b7051cf8e9bfea53/raw/77aec417f54ed3d996eb340f826de9d7569088f5/gistfile1.txt
This is not about socket directories, and SSH_AUTH_SOCK is set properly (as
demonstrated by the fact that dsa and rsa works).
mkdir /run/user/YOURUID
/run is not a thing on macOS. /var/run is but requires root access. The default
socket location on macOS is
iMac-TMP:~ joe$ ls -al /tmp/com.apple.launchd.OWt2QNcw7V total 0 drwx------ 2 joe wheel 102 Mar 1 21:13 . drwxrwxrwt 76 root wheel 2686 Mar 2 00:44 .. srw-rw-rw- 1 joe admin 0 Mar 1 21:13 Listeners
One such directory and socket are created for each user when using ssh.
Note that I had tried hacking in /usr/local/var/run in place of your /run, and
creating the directory, etc., which the test did proceed to use but it didn't
resolve the problem. (I had done that before reporting here.)
Yup! And remember, that was only a <= 10.10 issue, whereas the new "fun" is
across the board, 10.10, 10.11, and 10.12 (probably <= 10.9 too but we don't
test those).
Oh cool, so that patch was effective :)
Your T2947 says otherwise. So does T2847. Can you please clarify that?
Sure. I'm referring to 2.1.18 with your subsequent fixes. See
https://github.com/Homebrew/homebrew-versions/commit/8243792932da5b7746bf229d4b63abd7592b21b3
which has the relevant patches (thank you) on the left side of the diff.
Thanks. Can you please run the test again with
make check BIN_PREFIX=/usr/local/Cellar/gnupg21/2.1.19
2.1.18 did not have this problem, and the tests completed successfully within
the sandboxed Homebrew build environment.
Your T2947 says otherwise. So does T2847. Can you please clarify that?
This is the interesting part of the log:
Importing ssh keys...
> dsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
rsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
ecdsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
error fetching identities for protocol 1: agent refused operation
error fetching identities for protocol 2: agent refused operation
So, what happens for 'dsa', 'rsa', 'ecdsa' is that first the key is added, then
with ssh-add -l we check that it is in fact added. This works for dsa and rsa,
but fails for ecdsa.
Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?
This seems indeed a different problem than 2979.
SSH_AUTH_SOCK seems not to be set. I would suggest to try
mkdir /run/user/YOURUID
chown YOURUID /run/user/YOURUID
and try again.
As of e064c75b08a523f738108428fe0c417a46e66238 newlines are always escaped.
macOS 10.10, 10.11, 10.12
They're nearly the same, but T2980 has the workaround for this issue in
place (running make install first), so that it's clear the ssh-import.scm
problem is an independent issue.
Let's keep this one open to track missing options.
Justus, thanks for this work, it's great!. If we can solve the problem by doing
more clever socket(7) manipulation, that would be a big win.
How do you propose dealing with the getsockname() variations? or should we just
forbid the use of getsockname() entirely in the gnupg codebase?
dkg, I understand that GnuPG does not work with such a homedir, however, it is
not the act of creating the socket that is problematic. In fact, both
bind(2)ing and connect(2)ing is ok if one uses relative paths, as demonstrated
by the test program I have attached here.
Here is the program binding and connecting to a socket with an absolute path
length of ~10 * sizeof sockaddr_un.sun_path:
System: OpenBSD:6.0:GENERIC.MP#1992
sizeof addr.sun_path: 104
Running test with strlen (cwd): 22, name: '/tmp/test-unix-sockets/socket'
getsockname returned '/tmp/test-unix-sockets/socket', addrlen: 106
Running test with strlen (cwd): 22, name: 'socket'
getsockname returned 'socket', addrlen: 106
Running test with strlen (cwd): 126, name: 'socket'
getsockname returned 'socket', addrlen: 106
Running test with strlen (cwd): 1062, name: 'socket'
getsockname returned 'socket', addrlen: 106
This works on all Unices that I have access to. I've asked on gnupg-devel@ for
people to run it elsewhere.
I understand that '--create-socketdir' solves problems besides this one. But I
disagree with the statement that our handling of socket paths is unproblematic
because --create-socketdir solves this problem.
The --hostable option is a debugging aid and only used manually.
The nsswitch items "mymachine", "resolve", and "myhostname" are not known to
libdns but should have been skipped. "files" is the first entry and should have
delivered the result.
Fixed in cd32ebd152a522e362469ab969d91f8d49f28a60.
Seems that libdns does not pick it up /etc/hosts
Fix pushed. Thanks.
Simply not implemented. Will be in 2.1.19
Thanks for reporting. Fixed in master for 2.1.19.
(sorry, I accidentally removed the attached while while editing the mime type)
Can we test whether /run is mounted on a tmpfs ?
should we assume that /run is always on a tmpfs but /var/run is a classical Unix
w/o a tmpfs? Or is it better to have a configure option.
I can imagine to agree to auto-create the directory on a tmpfs.
Yes, notmuch decided that they needed to workaround the situation anyway,
because they're in an environment that doesn't create the standard per-user
rundir. That doesn't seem like a great argument that gpg should also fail in
environments where the standard per-user rundir is available. I can demonstrate
a number of environments where gpg or its daemons will fail, but i don't think
any of them justify forcing gpg or its daemons to *also* fail when those
environments aren't present.
In answer to your nitpick, here is evidence that gpg's daemons cannot create
their sockets when the GNUPGHOME is too long:
1 dkg@alice:~$ mkdir -m 0700
/home/dkg/tmp/very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-long
0 dkg@alice:~$
GNUPGHOME=/home/dkg/tmp/very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-very-long
gpgconf --launch dirmngr
gpgconf: error running '/usr/bin/gpg-connect-agent': exit status 1
gpgconf: error running '/usr/bin/gpg-connect-agent --dirmngr NOP': General error
1 dkg@alice:~$
FYI: It is fixed in 2.1.
Backporting the change to 2.0 will be a bit large, and I hesitate to do that.
Simple workaround is having multiple readers...
Most card readers only support a single card.
(This is the reason why it is not yet implemented.)
Could you please let us know the reader which supports multiple cards?
I was actually surprised to find out this doesn't already work...
I would like to be able to have two or more GnuPG cards inserted at the same
time, and have gnupg/gpg-agent/scdaemon notice all of them and use whichever one
was appropriate for the operation at hand, without my having to switch them in
and out.
My personal application for this is that I have a personal key and a work key,
and I want to be able to sign with either of them without having to swap
hardware around. It's pretty easy to set up all the other parts of this to be
seamless with Thunderbird/Enigmail/gnupg2... it works fine until you move the
keys to cards, at which point gnupg's inability to automatically choose the card
it needs really shows up.
In an ideal world, this would also work with gpg-agent as a backend for ssh.
Notmuch deemed --create-socketdir to be insufficient for their test suite:
https://notmuchmail.org/pipermail/notmuch/2017/024148.html
Now they create GNUPGHOMEs in /tmp. That is exactly what our test suite does.
(We also use --create-socketdir, but we don't rely on it, and indeed, on my
system it fails b/c the per-user directory is not created. Likewise on the
OpenBSD build server, and the macOS one.)
Nitpick: You wrote:
when GNUPGHOME points to a directory whose path is larger than
sockaddr_un.sun_path, daemons like gpg-agent and dirmngr cannot create their
sockets.
I don't think this is correct. I have not seen any evidence that creating the
socket is problematic.
As of d379a0174cca595204b32da9a66c513a1304e6d0 auto-key-retrieve is configurable.
As of ebeccd73eb85f9027f0985d77dfe901266c6ddef the trust model is configurable
via gpgconf.
ntbtls support is now available in master and we will release a TLS enabled
2.1.19 installer for Windows.
Right now it is somewhat limited and does not work with some sites, notably
those which allow only ECC ciphersuites. An example for such a site is
posteo.de. Note that posteo.net sends a a bogus certifcate with rediretion to
posteo.de.
Most other sites work.
Ubuntu uses a bad combination of an older gpg version and a more current
libgcrypt version. We can't do anything about it. Someone may want to escalate
this to Ubuntu; they should definitely get an update out.