fwiw, i'm seeing this too, over at https://bugs.debian.org/846953 , for a user
with an insanely large (10240-bit) RSA key when it is locked with a passphrase.

I'm attaching such an example secret key (with passphrase "abc123"), and you can
trigger the crash with:

gpg --batch --yes --import test-hugekey.key
echo test | gpg -r 861A97D02D4EE690A125DCC156CC9789743D4A89

--encrypt --armor --trust-model=always --batch --yes --output data.gpg

            gpg --decrypt data.gpg

While i think it's fair to say that we need to have some limits on the sizes of
keys we can handle, gpg-agent should not crash when asked to deal with
extra-large keys, it should fail gracefully and return a sensible error code.

I've updated the patch series here to the series we're using in debian for 2.1.16.

D389: 918_0005-dirmngr-Drop-useless-housekeeping.patch

D390: 917_0004-dirmngr-Avoid-automatically-checking-upstream-swdb.patch

D391: 916_0003-dirmngr-Lazily-launch-ldap-reaper-thread.patch

D392: 915_0002-dimrngr-Avoid-need-for-hkp-housekeeping.patch

D393: 914_0001-dirmngr-hkp-Avoid-potential-race-condition-when-some.patch

In practice, dirmngr from git master still wakes up every few seconds due to the
ldap-reaper thread, even if no connections to ldap have ever happened.

the patch dirmngr-Lazily-launch-ldap-reaper-thread.patch avoids this additional
wakeup at least for those dirmngr instances that have never used LDAP.

We're shipping these patches in debian unstable as of 2.1.15-9.

D394: 912_0145-dirmngr-Lazily-launch-ldap-reaper-thread.patch

D395: 911_0144-dirmngr-Drop-useless-housekeeping.patch

D396: 910_0143-dimrngr-Avoid-need-for-hkp-housekeeping.patch

D397: 909_0142-dirmngr-hkp-Avoid-potential-race-condition-when-some.patch

D398: 908_0141-dirmngr-More-win32-system-daemon-cleanup.patch

I submitted these patches on the gnupg-devel mailing list in November 2016:

https://lists.gnupg.org/pipermail/gnupg-devel/2016-November/032011.html

These are working for me to keep gpg-agent idle on platforms that support
inotify when the user doesn't use scdaemon, and we're now shipping with them
applied in debian unstable.

D276: 907_0140-agent-Avoid-scheduled-checks-on-socket-when-inotify-.patch

D277: 906_0139-agent-Avoid-tight-timer-tick-when-possible.patch

D278: 905_0138-agent-Allow-threads-to-interrupt-main-select-loop-wi.patch

D279: 904_0137-agent-Create-framework-of-scheduled-timers.patch

I'm also seeing this behavior when there is something wrong with the reverse DNS
lookups. For example:

Nov 08 10:54:36 alice dirmngr[1714]: handler for fd 5 started
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> # Home: /home/dkg/.gnupg
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> # Config:
/home/dkg/.gnupg/dirmngr.conf
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK Dirmngr 2.1.15 at your
service
Nov 08 10:54:36 alice dirmngr[1714]: connection from process 7623 (1000:1000)
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- GETINFO version
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> D 2.1.15
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- KEYSERVER
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> S KEYSERVER
hkps://hkps.pool.sks-keyservers.net
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- KS_GET --
0x2E8DD26C53F1197DDF403E6118E667F1EB8AF314
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L3: ASSERT:
mpi.c[_gnutls_x509_read_uint]:246
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]:
Allocating epoch #0
Nov 08 10:54:36 alice dirmngr[1714]: can't connect to 'oteiza.siccegge.de':
Invalid argument
Nov 08 10:54:36 alice dirmngr[1714]: error connecting to
'https://oteiza.siccegge.de:443': Invalid argument
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]: Start
of epoch cleanup
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]: End of
epoch cleanup
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]: Epoch
#0 freed
Nov 08 10:54:36 alice dirmngr[1714]: command 'KS_GET' failed: Invalid argument
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> ERR 167804976 Invalid
argument <Dirmngr>
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- BYE
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK closing connection
Nov 08 10:54:36 alice dirmngr[1714]: handler for fd 5 terminated

This appears to be because the pool included 92.43.111.21, which has a PTR of
oteiza.siccegge.de, despite the fact that oteiza.siccegge.de has no A record.

There is no reason for dirmngr to be talking to the member of the pool by its
hostname, anyway -- it should make the connection by IP address, with the TLS
SNI set to the pool name.

D386: 903_0001-gnome3-Fall-back-to-curses-if-screensaver-is-locked.patch

Attached is a patch to check for locked screensaver and fall back to curses if
detected.

Perhaps gcr needs to refuse to prompt in the event that the graphical session is
known-idle/locked (in screensaver mode, whatever). Then the pinentry could know
to fall back to the tty because of the locked screen.

I just spent a while trying to research this, and i'm afraid that the code i've
written to detect whether gcr is available does nothing to detect whether the
screen is currently locked.

Furthermore, when "getpin" is called against a dbus session that is locked, it
immediately returns with a "Cancelled" message, in a way that is pretty
difficult to diagnose.

However, it looks like i can query the gnome screensaver via dbus to see whether
the screen is locked. From the command line, that's:

dbus-send --print-reply=literal --session --dest=org.gnome.ScreenSaver

/org/gnome/ScreenSaver org.gnome.ScreenSaver.GetActive

which returns a boolean true or false depending on whether the screen is locked.

We'd just need to translate it into GDBus, i think, perhaps using something
higher-level like g_dbus_connection_call(), or something lower-level, like
g_dbus_connection_send_message_with_reply() (or their synchronous variants):

file:///usr/share/doc/libglib2.0-doc/gio/GDBusConnection.html#g-dbus-connection-call
file:///usr/share/doc/libglib2.0-doc/gio/GDBusConnection.html#g-dbus-connection-send-message-with-reply

In your example, i don't think updatestartuptty is necessary for text-mode
prompting -- the "gpg --decrypt …" process will be able to detect which tty it
is connected to and pass it to the agent.

But the question here has to do with graphical consoles as well, and i don't
think there's a clear answer yet.

There are two X11 graphical sessions in the example:

a) the local machine's graphical console, where the user is currently sitting,
running ssh *to* the remote machine
b) the remote machine's graphical console, where the user is logged in, but idle

There are also three kinds of pinentry user-attention-getting mechanisms:

0) terminal

1. X11
2. d-bus

finally, i'll note that there are (at least) two d-bus user sessions running in
this example: on the remote host and on the local host. I'm assuming in this
example that the user has a single shared d-bus session across all logins on the
computer (this is the dbus-user-session model, which is well-aligned with the
gpg-agent standard-socket model, where there is one running process per user per
machine)

Since "ssh -X remote" forwards the X11 session but not the d-bus session, any
d-bus-based pinentry (like pinentry-gnome3) will connect to the d-bus session on
the remote machine. But the d-bus session on the remote machine is *also*
connected to the remote graphical (X11) console.

pinentry on the remote machine has two choices:

x) talk to the d-bus session it is connected to (which will trigger a prompt on
the remote graphical console, or
y) fall back to curses

If it chooses (x) then the user is unlikely to see the prompt (they're not
sitting in front of that graphical console). But it's not clear how to
distinguish the situation from normal use in order to choose (y).

Perhaps gcr needs to refuse to prompt in the event that the graphical session is
known-idle/locked (in screensaver mode, whatever). Then the pinentry could know
to fall back to the tty because of the locked screen. If it does that, then the
error case (where the graphical prompt is shown on the idle session) is limited
to situations where the user left the remote graphical console unlocked. I
don't know whether we can get gcr to report that successfully or not, though.

How many people has this happened to? how many people haven't known to find you
on freenode and ask about it? how many people have just given up on gpg
instead, or just decided "2.1 is broken"?

Shouldn't we fix this for them?

I like this work, thanks for it! I wonder whether it would also be useful for
full-match userID, not only for a raw e-mail address?

For example, if i query for '=Peter Palfrader' or '=ssh://host.example', it
ought to give me the key with the highest-validity binding for the requested
user ID.

(see on-list discussion at
https://lists.gnupg.org/pipermail/gnupg-users/2016-October/056978.html)

I'm trying to understand this, but I'm not seeing it.

Here's the test i did. While recording all traffic from my machine on port 53
(the dns port), i ran:

    GNUPGHOME=$(mktemp -d) gpg-connect-agent --dirmngr

That interactive session looked like this:

> getinfo dnsinfo
OK - ADNS w/o Tor support
> getinfo tor
dirmngr[11713.1]: command 'GETINFO' failed: False
ERR 167772416 False <Dirmngr> - Tor mode is NOT enabled
> keyserver --clear
OK
> keyserver hkps://hkps.pool.sks-keyservers.net
OK
> keyserver --resolve hkps://hkps.pool.sks-keyservers.net
dirmngr[11713.1]: DNS query returned an error or no records: No such domain

(nxdomain)

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'bone.digitalis.org'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'ip-209-135-211-141.ragingwire.net'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'gpg.nebrwesleyan.edu'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'host-37-191-220-247.lynet.no'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'cryptonomicon.mit.edu'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'zimmerman.mayfirst.org'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'sks.srv.dumain.com'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'b4ckbone.de'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'sks.spodhuis.org'

dirmngr[11713.1]: resolve_dns_addr for 'hkps.pool.sks-keyservers.net':

'oteiza.siccegge.de'

    S # https://cryptonomicon.mit.edu:443
    OK
    > keyserver --hosttable
    S # hosttable (idx, ipv6, ipv4, dead, name, time):
    S #   0       hkps.pool.sks-keyservers.net
    S #   .   --> 8 1 5* 3 4 2 10 9 7 6
    S #   1   4   bone.digitalis.org v4=212.12.48.27
    S #   2   4   ip-209-135-211-141.ragingwire.net v4=209.135.211.141
    S #   3   4   gpg.nebrwesleyan.edu v4=192.94.109.73
    S #   4   4   host-37-191-220-247.lynet.no v4=37.191.220.247
    S #   5   4   cryptonomicon.mit.edu v4=18.9.60.141
    S #   6   4   zimmerman.mayfirst.org v4=216.66.15.2
    S #   7   4   sks.srv.dumain.com v4=85.119.82.209
    S #   8   4   b4ckbone.de v4=193.164.133.100
    S #   9   4   sks.spodhuis.org v4=94.142.242.225
    S #  10   4   oteiza.siccegge.de v4=92.43.111.21
    OK
    >

So, the SRV lookup did indeed fail, but subsequent queries succeeded.

I've attached a pcapng file of the network traffic sent and received from the
described test.

The textual version of the traffic is:

query 0x311f SRV _hkp._tcp.hkps.pool.sks-keyservers.net
query response 0x311f No such name SRV

_hkp._tcp.hkps.pool.sks-keyservers.net SOA ns2.kfwebs.net

query 0x3120 A hkps.pool.sks-keyservers.net
query response 0x3120 A hkps.pool.sks-keyservers.net A 92.43.111.21 A

94.142.242.225 A 193.164.133.100 A 85.119.82.209 A 216.66.15.2 A 18.9.60.141 A
37.191.220.247 A 192.94.109.73 A 209.135.211.141 A 212.12.48.27

query 0xbd61 PTR 27.48.12.212.in-addr.arpa
query response 0xbd61 PTR 27.48.12.212.in-addr.arpa PTR bone.digitalis.org
query 0x384a PTR 141.211.135.209.in-addr.arpa
query response 0x384a PTR 141.211.135.209.in-addr.arpa PTR

ip-209-135-211-141.ragingwire.net

query 0xb36e PTR 73.109.94.192.in-addr.arpa
query response 0xb36e PTR 73.109.94.192.in-addr.arpa PTR gpg.nebrwesleyan.edu
query 0xcac3 PTR 247.220.191.37.in-addr.arpa
query response 0xcac3 PTR 247.220.191.37.in-addr.arpa PTR

host-37-191-220-247.lynet.no

query 0xd28b PTR 141.60.9.18.in-addr.arpa
query response 0xd28b PTR 141.60.9.18.in-addr.arpa PTR cryptonomicon.mit.edu
query 0x4be9 PTR 2.15.66.216.in-addr.arpa
query response 0x4be9 PTR 2.15.66.216.in-addr.arpa CNAME

2.0-27.15.66.216.in-addr.arpa PTR zimmerman.mayfirst.org PTR zimmermann.mayfirst.org

query 0x823b PTR 209.82.119.85.in-addr.arpa
query response 0x823b PTR 209.82.119.85.in-addr.arpa PTR sks.srv.dumain.com
query 0x3b0c PTR 100.133.164.193.in-addr.arpa
query response 0x3b0c PTR 100.133.164.193.in-addr.arpa PTR b4ckbone.de
query 0x9600 PTR 225.242.142.94.in-addr.arpa
query response 0x9600 PTR 225.242.142.94.in-addr.arpa PTR sks.spodhuis.org
query 0xed36 PTR 21.111.43.92.in-addr.arpa
query response 0xed36 PTR 21.111.43.92.in-addr.arpa PTR oteiza.siccegge.de

thanks, that seems to have resolved the problem in my tests.

I'm attaching the lsof and strace transcript in text form so it can be read
without linebreaks

This is apparently just re-reported on gnupg-users:

https://lists.gnupg.org/pipermail/gnupg-users/2016-October/056892.html

So i don't think it's fixed.

And fwiw, it seems like a clear bug to me if i use "ssh-add" and then it is not
added to the agent.

From the ssh-add's client's perspective, some keys are magically never added,
but others are. This kind of mystery behavior is confusing and frustrating. If
gpg-agent is going to handle the ssh-agent protocol, it should aim toward behave
as the user of the ssh-agent protocol expects, regardless of whether the user
knows that they're using gpg-agent or some other implementation.

another item for consistency is gpg-agent's different behavior between
--enable-ssh-socket and --extra-socket (and the undocumented --browser-socket,
for that matter, but since it's not documented maybe it's fine to just change
that one).

Agreed, but i ran into this while looking at python-gnupg, which is now failing
when using GnuPG 2.1. so they're facing breakage either way. It would be
better to have all current releases doing the expected behavior than to imagine
that we can bump this variance in behavior along indefinitely.

So i see a couple options:

a) We import a secret key -- this requires that we launch the agent to store it.
b) We import a public key and see that its preferences do match our
implementation -- in this case, we don't need to talk to the agent, right?
c) We import a public key and see that its preferences do not match our
implementation -- in this case, we could check whether the agent has the
corresponding secret key, and if it does, we could complain to the user.

instead of (c), though, we could trigger such a test the other way around: if
we're using a secret key and we notice that its public preferences don't match
our implementation, that's when we could warn the user about the mismatch.

So i've tested this locally with:

    export GNUPGHOME=$(mktemp -d)
    gpg --quick-gen-key 'test user <test@example.org>'
    gpg --armor --export-secret-key 'test user <test@example.org>'

(choosing no passphrase during the prompts that come up during the quick-gen-key
step). The final export step works fine.

Can you show what steps you're taking that fail for you, Andre?

sure: using the attached "dkg.gpg" file (a pruned version of my own public key),
i did:

if --list-config is deprecated, should it emit a warning? doc/gpg.texi shows no
mention that it is deprecated, or that "gpgconf --list-options gpg" should be
preferred.

Also, i note that --list-config is still used in the test suite:

tests/openpgp/defs.inc uses it with "ciphername" and "digestname", and
tests/openpgp/defs.scm uses it with "ciphername" and "digestname" and
"pubkeyname". I don't see any way to get the same information out of gpgconf.
Perhaps gpgconf needs to provide some equivalent?

hm, there is also:

    gpgconf --list-options gpg | \
         awk -F: '/^default-key:/{ print $10 }'

It's not clear to me when anyone should use "gpgconf --list-options gpg" and
when they should use "gpg --with-colons --list-config".

Is there some place where one or the other is more important?