Duplicate of T2962

I think it is easier to enforce this than to handle bug reports due to
export/import and whatever problems.

We should better fix that by adding a new API to libassuan so that we have that
code only once.

Thanks for the report.

Shall I then thake this bug?

That implicit local is for backward compatibility and to avoid network lookups
as much as possible (privacy leak). "clear" is required because auto-key-locate
is cumulative.

I doubt that this is Windows only. On Linux we use our own driver but on
Windows we have to resort to PC/SC. My educated guess is that we are in some
blocking system call which is not npth_unprotected.

Fixed in 0c4d0620d327e8a2069532a5519afefe867a47d6.

So I went over the code that does --locate-key. There, the available methods
are ordered, and if 'local' is not given, it is explicitly done first, unless
'nodefault' is given. This is one of the parts of GnuPG that I'm really afraid
to change ;)

I have to refine my statement. We store the 'ultimateley trusted flag in the
trustdb and thus we require a trustdb when creating a new key. That is so that
we know the key has been created by us and is not an imported key.
Thus for most commands the trustdb should not be created but for key generation
it is better to safe that ultimately trusted flag in the trustdb.

Hum, there is something strange going on here:

% gpg --auto-key-locate wkd --locate-key root@leckerlecker
... no update...
% gpg --auto-key-locate wkd,local --locate-key root@leckerlecker
... no update...
% gpg --auto-key-locate clear,wkd,local --locate-key root@leckerlecker
... update!...

Thanks to you and thanks to Niibe in advance :)

Glad to hear that. Niibe will have a closer look at the issue tomorrow.

Fixed in 4735ab96aa5577d40ba7b3f72d863057198cc6a7.

And it has now passed CI:
https://github.com/Homebrew/homebrew-versions/pull/1536
https://bot.brew.sh/job/Homebrew%20Versions%20Pull%20Requests/1824/

So I've merged the PR.

The patch works!

I read the code and documentation for key-edit's revuid, and --quick-revoke-uid,
and the code makes no effort to ensure that one valid UID remains.

I read rfc4880 trying to find the basis for "some non-revoked User ID must
remain", but the only justification I could find is in section 11.1.,
Transferable Public Keys, that states that at least one UID must be included if
one wants to transfer keys.

So, do we actually want to enforce that or fix the documentation?

A possible workaround for you is to disable the use of scdaemon in the tests.
Currently, it is not needed anyway. I'll attach a patch to do that.

D412: 966_issue2980-workaround.patch

This is interesting indeed. Might be related to a recent change to support
multiple smart cards.

Here's the referenced crash file:
https://gist.github.com/ilovezfs/ebfccf2515fc7d7952edcc4c13ff8013

Here's what's happening in the system log when the failed test runs:

Mar  2 03:06:53 iMac-TMP com.apple.xpc.launchd[1] (com.apple.auditd[39537])
<Warning>: Endpoint has been activated through legacy launch(3) APIs. Please
switch to XPC or bootstrap_check_in(): com.apple.auditd
Mar  2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Notice>:
SecTaskLoadEntitlements failed error=22
Mar  2 03:06:53 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed
PCSC.framework client without com.apple.security.smartcard entitlement
Mar  2 03:06:53 iMac-TMP joe[39540] <Warning>: audit warning: soft /var/audit
Mar  2 03:06:53 iMac-TMP joe[39541] <Warning>: audit warning: allsoft 
Mar  2 03:06:54 iMac-TMP joe[39544] <Warning>: audit warning: closefile
/var/audit/20170302110453.20170302110653
Mar  2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Notice>:
SecTaskLoadEntitlements failed error=22
Mar  2 03:06:54 iMac-TMP com.apple.ctkpcscd[39536] <Warning>: Refusing sandboxed
PCSC.framework client without com.apple.security.smartcard entitlement
Mar  2 03:06:54 iMac-TMP com.apple.xpc.launchd[1] (com.apple.ReportCrash[39548])
<Warning>: Endpoint has been activated through legacy launch(3) APIs. Please
switch to XPC or bootstrap_check_in(): com.apple.ReportCrash
Mar  2 03:06:54 iMac-TMP ReportCrash[39548] <Notice>: Saved crash report for
scdaemon[39535] version 0 to
/Users/joe/Library/Logs/DiagnosticReports/scdaemon_2017-03-02-030654_iMac-TMP.crash

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

It's the system default. There's no other version of ssh that gets installed.
Our own ssh formula is homebrew/dupes/openssh and is explicitly barred from
being used as a dependency by anything else, as is anything else in homebrew/dupes.

10.12.3
robotunicorn ~ # ssh -V
OpenSSH_7.3p1, LibreSSL 2.4.1

10.11.6
iMac-TMP:~ joe$ ssh -V
OpenSSH_6.9p1, LibreSSL 2.1.8

yosemitevm ~ # ssh -V
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011

Regarding the sandbox, here's where it's implemented:
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/sandbox.rb

It's invoked as

/usr/bin/sandbox-exec -f /tmp/homebrew20170302-24230-1xmlw7l.sb nice
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -W0 -I
/usr/local/Homebrew/Library/Homebrew --
/usr/local/Homebrew/Library/Homebrew/build.rb
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-versions/gnupg21.rb

The contents of the .sb file are as follows:

iMac-TMP:~ joe$ cat /tmp/homebrew20170302-24230-1xmlw7l.sb
(version 1)
(debug deny) ; log all denied operations to /var/log/system.log
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/private/var/tmp"))
(allow file-write* (regex #"^/private/var/folders/[^/]+/[^/]+/[C,T]/"))
(allow file-write* (subpath "/private/tmp"))
(allow file-write* (subpath "/Users/joe/Library/Caches/Homebrew"))
(allow file-write* (subpath "/Users/joe/Library/Logs/Homebrew/gnupg21"))
(allow file-write* (subpath "/Users/joe/Library/Developer"))
(allow file-write* (subpath "/usr/local/Cellar/gnupg21"))
(allow file-write* (subpath "/usr/local/etc"))
(allow file-write* (subpath "/usr/local/var"))
(allow file-write*
    (literal "/dev/ptmx")
    (literal "/dev/dtracehelper")
    (literal "/dev/null")
    (literal "/dev/zero")
    (regex #"^/dev/fd/[0-9]+$")
    (regex #"^/dev/ttys?[0-9]*$")
    )
(deny file-write*) ; deny non-whitelist file write operations
(allow process-exec
    (literal "/bin/ps")
    (with no-sandbox)
    ) ; allow certain processes running without sandbox
(allow default) ; allow everything else

The environment variables themselves are not different between sandboxed and
non-sandboxed builds.

From what I've seen there is no variation in getsockname, it just returns
whatever path is passed to bind. I don't understand the need for getsockname
tbh, because we are the ones that bind the socket in the first place.

(The only variation seems to be that the function is broken on Hurd...).

Fixed in 62d21a4ab4029b32ea129f1cf3a0e1f22e2fb7b0.

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

(I'm mildly annoyed that I have to ask again. You are not forthcoming with
information about your environment. macOS is somewhat alien for us, and if you
want help with tracking down the issue, you need to give us more information.
Note that we really do make an effort, and we have a macOS build slave that does
not see these issues:

https://jenkins.gnupg.org/job/gnupg/XTARGET=native,label=macos/

(though we get our build dependencies from pkgsrc, and you get it from your
packages I guess, so there are bound to be differences).)

From your latest log I see that the version of ssh used supports ed25519, so
this must be version newer than 6.5.

I just committed a patch that dumps the ssh version and the path to the binary
when executing the test:

https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commitdiff;h=74cb3b230c1f99afc5fd09bccc24186a63b154b0

D413: 965_gnupg-2.1.19-solaris-ucred.patch

Here's the successful log with --no-sandbox:
https://gist.githubusercontent.com/ilovezfs/a886421569e625d0b7051cf8e9bfea53/raw/77aec417f54ed3d996eb340f826de9d7569088f5/gistfile1.txt

This is not about socket directories, and SSH_AUTH_SOCK is set properly (as
demonstrated by the fact that dsa and rsa works).

mkdir /run/user/YOURUID

/run is not a thing on macOS. /var/run is but requires root access. The default
socket location on macOS is

iMac-TMP:~ joe$ ls -al /tmp/com.apple.launchd.OWt2QNcw7V
total 0
drwx------   2 joe   wheel   102 Mar  1 21:13 .
drwxrwxrwt  76 root  wheel  2686 Mar  2 00:44 ..
srw-rw-rw-   1 joe   admin     0 Mar  1 21:13 Listeners

One such directory and socket are created for each user when using ssh.

Note that I had tried hacking in /usr/local/var/run in place of your /run, and
creating the directory, etc., which the test did proceed to use but it didn't
resolve the problem. (I had done that before reporting here.)

Yup! And remember, that was only a <= 10.10 issue, whereas the new "fun" is
across the board, 10.10, 10.11, and 10.12 (probably <= 10.9 too but we don't
test those).

Oh cool, so that patch was effective :)

Your T2947 says otherwise. So does T2847. Can you please clarify that?

Sure. I'm referring to 2.1.18 with your subsequent fixes. See
https://github.com/Homebrew/homebrew-versions/commit/8243792932da5b7746bf229d4b63abd7592b21b3
which has the relevant patches (thank you) on the left side of the diff.

Thanks. Can you please run the test again with

make check BIN_PREFIX=/usr/local/Cellar/gnupg21/2.1.19

2.1.18 did not have this problem, and the tests completed successfully within
the sandboxed Homebrew build environment.

Your T2947 says otherwise. So does T2847. Can you please clarify that?

This is the interesting part of the log:

Importing ssh keys...

> dsa Executing: '/usr/bin/ssh-add' '-'

Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
rsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
ecdsa Executing: '/usr/bin/ssh-add' '-'
Identity added: (stdin) ((stdin))
Executing: '/usr/bin/ssh-add' '-l' '-E' 'md5'
error fetching identities for protocol 1: agent refused operation
error fetching identities for protocol 2: agent refused operation

So, what happens for 'dsa', 'rsa', 'ecdsa' is that first the key is added, then
with ssh-add -l we check that it is in fact added. This works for dsa and rsa,
but fails for ecdsa.

Can you please give us ssh -V, and describe the sandbox environment? Does it
affect which ssh version is used?

This seems indeed a different problem than 2979.

SSH_AUTH_SOCK seems not to be set. I would suggest to try

mkdir /run/user/YOURUID
chown YOURUID /run/user/YOURUID

and try again.

As of e064c75b08a523f738108428fe0c417a46e66238 newlines are always escaped.

macOS 10.10, 10.11, 10.12

They're nearly the same, but T2980 has the workaround for this issue in
place (running make install first), so that it's clear the ssh-import.scm
problem is an independent issue.

Duplicate of T2980

Please describe your platform.
I guess the log is the same as in T2980, thus I will merge them.

This happens on at least PPC Mac OS X 10.4.11, Tiger. Compiler is by default Apple's
version of GCC 4.2. The error is reported as this:

libtool: compile: /opt/local/bin/gcc-apple-4.2 -DHAVE_CONFIG_H -I. -I..
-I.. -I/opt/local/include -I/opt/local/include -pipe -Os -arch ppc -Wall
-Wcast-align -Wshadow -Wstrict-prototypes -Wpointer-arith -MT
libassuan_la-assuan-socket.lo -MD -MP -MF .deps/libassuan_la-assuan-
socket.Tpo -c assuan-socket.c -fno-common -DPIC -o .libs/libassuan_la-
assuan-socket.o
assuan-socket.c: In function 'socks5_connect':
assuan-socket.c:732: error: 'INADDR_LOOPBACK' undeclared (first use in
this function)
assuan-socket.c:732: error: (Each undeclared identifier is reported only
once
assuan-socket.c:732: error: for each function it appears in.)
make[3]: * [libassuan_la-assuan-socket.lo] Error 1
make[3]: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3/src'
make[2]: * [all] Error 2
make[2]: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3/src'
make[1]: * [all-recursive] Error 1
make[1]: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3'
make: * [all] Error 2
make: Leaving directory
`/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3'
Command failed: cd
"/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.or
g_release_tarballs_ports_devel_libassuan/libassuan/work/libassuan-2.4.3"
&& /usr/bin/make -w all

Cause is that the declaration of 'INADDR_LOOPBACK' is hidden behind some guards. The
two external links document the situation, they also offer patches, either the attached
one or this addition for src/assuan-socket.c:

//fix missing define in MacOSX 10.4 Tiger
#ifndef INADDR_LOOPBACK

#define INADDR_LOOPBACK (u_int32_t)0x7f000001

#endif

In ten days I'll be at home again with my Apple PowerBook G4 and might have time to
check the situation in Mac OS X 10.5, Leopard.

D380: 963_src_assuan-socket_INADDR_LOOPBACK.patch

Justus, thanks for this work, it's great!. If we can solve the problem by doing
more clever socket(7) manipulation, that would be a big win.

How do you propose dealing with the getsockname() variations? or should we just
forbid the use of getsockname() entirely in the gnupg codebase?

See T2910.

I addressed this for GPGME in 60273e8b2c11d42215a5707bc55e3e0d8f350e07 but
apparently forgot to mention that here.

I'll keep the bug open until I fixed this in all packages.

I added the following snippet to our pound configuration in the ListenHTTP
section for IPv4:

1. Justus: Redirect all jenkins request to https. Service HeadRequire "Host:.*jenkins.gnupg.org" Redirect 301 "https://jenkins.gnupg.org" End

I hope I didn't break anything. Jenkins is much nicer to use now :)

Wichtiger Hinweis:
Diese E-Mail enthält vertrauliche oder rechtlich geschützte Informationen.
Wenn Sie nicht der beabsichtigte Empfänger sind, informieren Sie bitte sofort den Absender und löschen Sie diese E-Mail. Das unbefugte Kopieren dieser E-Mail oder die unbefugte Weitergabe der enthaltenen Informationen ist nicht gestattet.

Thanks for your report. Indeed it should work as you described and we have code
in the installer to print a non admin warning. If this is not shown then it is a
bug.

On a related note: I have on my TODO list to enable "Single User" installation
in case a user tries to install Gpg4win without admin rights, because with the
modern gnupg versions we don't need admin rights anymore. Would this also have
solved your problem but or do you specifically want to have Gpg4win installed
systemwide?

Yes, it's the same issue.

dkg, I understand that GnuPG does not work with such a homedir, however, it is
not the act of creating the socket that is problematic. In fact, both
bind(2)ing and connect(2)ing is ok if one uses relative paths, as demonstrated
by the test program I have attached here.

Here is the program binding and connecting to a socket with an absolute path
length of ~10 * sizeof sockaddr_un.sun_path:

System: OpenBSD:6.0:GENERIC.MP#1992
sizeof addr.sun_path: 104
Running test with strlen (cwd): 22, name: '/tmp/test-unix-sockets/socket'

getsockname returned '/tmp/test-unix-sockets/socket', addrlen: 106

Running test with strlen (cwd): 22, name: 'socket'

getsockname returned 'socket', addrlen: 106

Running test with strlen (cwd): 126, name: 'socket'

getsockname returned 'socket', addrlen: 106

Running test with strlen (cwd): 1062, name: 'socket'

  getsockname returned 'socket', addrlen: 106

This works on all Unices that I have access to. I've asked on gnupg-devel@ for
people to run it elsewhere.

I understand that '--create-socketdir' solves problems besides this one. But I
disagree with the statement that our handling of socket paths is unproblematic
because --create-socketdir solves this problem.

Isn't this the same as T2975 ?

The --hostable option is a debugging aid and only used manually.

The nsswitch items "mymachine", "resolve", and "myhostname" are not known to
libdns but should have been skipped. "files" is the first entry and should have
delivered the result.

Fixed in cd32ebd152a522e362469ab969d91f8d49f28a60.

Seems that libdns does not pick it up /etc/hosts

Fix pushed. Thanks.

Simply not implemented. Will be in 2.1.19