Oh well, using a curl based key server helper this might have worked in the
past. We better implement that for 2.2

There has never been support in GnuPG for https via an http proxy.
So can we change this to a feature request?

I guess the best solution is to handle this the same way as a missing
nsswitch file. Here is a non-tested patch; for a quick test the
change of the condition is sufficient.

diff --git a/dirmngr/dns-stuff.c b/dirmngr/dns-stuff.c
index f0de357..956fe72 100644

• a/dirmngr/dns-stuff.c

+++ b/dirmngr/dns-stuff.c
@@ -496,14 +496,15 @@ libdns_init (void)

fname = "/etc/nsswitch.conf";
err = libdns_error_to_gpg_error
  (dns_nssconf_loadpath (ld.resolv_conf, fname));

• if (err)

+ if (err || !ld.resolv_conf->lookup[0])

{

• log_error ("failed to load '%s': %s\n", fname, gpg_strerror (err));
• /* not fatal, nsswitch.conf is not used on all systems; assume
• * classic behavior instead. Our dns library states "bf" which tries
• * DNS then Files, which is not classic; FreeBSD
• * /usr/src/lib/libc/net/gethostnamadr.c defines default_src[] which
• * is Files then DNS, which is. */

+ if (err)
+ log_error ("failed to load '%s': %s\n", fname, gpg_strerror (err));
+ /* Not fatal, nsswitch.conf is not used on all systems;
+ * assume classic behavior instead. Note that some systemd
+ * based systems allow for custom keywords which are not
+ * known to us and thus lead to an empty result set; we then
+ * also fallback to classic behavior. */

if (opt_debug)
  log_debug ("dns: fallback resolution order, files then DNS\n");
ld.resolv_conf->lookup[0] = 'f';

I have seen that discussion and will takle care of this bug soon.

The unnecessary PTR lookup is causing problems for other people too, over on
https://bugs.debian.org/854359

Thanks for this work (and sorry to have just blindly/wrongly assumed that
--no-use-tor already existed without checking it).

On modern debian systems, the default tor daemon will always be listening on
unix domain socket /run/tor/socks. So a simple attempt to connect to that
socket should be sufficient -- it should fail immediately if the socket isn't
present or if no one is listening on it.

This seems cheap and fast enough to be able to do it on every query to me,
rather than introducing additional runtime state to dirmngr. just try to
connect, and if it doesn't work, fall back to a normal connection (you'd want to
do that anyway in case the tor daemon goes away after dirmngr had been launched).

Okay, that first part has been pushed. Now need to figure out how to test for
Tor in a clean way.

I will do some rework to make testing for tor easier ....

I think this is a good idea. If Tor is already running we can expect that the
user wants to use Tor as much as possible and thus tehre should be no need for
any configuration.

I do not think that we need a new option (except for making --no-use-tor). To
avoid checking for tor with every new connection to Dirmngr, I would do a test
at startup and after each reload.

for cases (1), (2), and (3) it sounds like you don't need the PTR at all. right?

For your case (4), i think we should reject hkps via literal IP addresses. It's
not a real-world use case, and if you want to test/experiment with hkps as a
developer, you should have at least the capacity to edit /etc/hosts (or whatever
your system's equivalent is). Anyway, trying to support this case for the
purposes of debugging doesn't make sense if support for this case is the cause
of the bugs in the first place ;)

re: duplicate hosts: I live in a part of the world where dual-stack
connectivity is sketchy at best. And, when connecting to things over Tor, it's
possible that connections to IPv4 hosts will have a different failure rate than
IPv6 connections.
So unless you already know that the host itself is down, why would you avoid
trying the other routes you have to it?

Look at it another way: when trying to reach host X, you discover that X has two
IP addresses, A and B. You try to reach A and it's not available. Isn't it
better to try B instead, rather than to avoid trying B at all just because A was
unreachable?

In a pool scenario, you might want to try to cluster addresses together by
perceived identity so that you can try an entirely different host first, rather
than a different address for the same host who happens to be in the pool twice.
But that strikes me as a very narrow optimization, certainly something that'd
only be worth implementing after we've squeezed the last bit of performance out
of other parts of the code (parallel connections, "happy eyeballs", etc).
Definitely not something to bother with at the outset. So i'd say drop that
optimization for simplicity's sake.

So the simplest approach is:

a) know the configured name of the keysserver
b) resolve it to a set of addresses
c) try to connect to those addresses, using the configured name of the server
for SNI and HTTP Host:

This is all that's needed for cases (1) and (3), and it could also be used in
case (2) if you see (b) as a two-stage resolution process (name→SRV→A/AAAA),
discarding the intermediate names from the SRV. Given that some people may
access the pool via case (1), and servers in the pool won't be able to
distinguish between how they were selected (SRV vs. A/AAAA), they'll still
accept the connections.

If you decide the additional complexity is worthwhile for tracking the
intermediate names in the SRV records, you can always propagate the intermediate
names wherever you like locally without changing the "simplest" algorithm.

If you really want to use the names from the SRV in collecting, then the
algorithm should change to:

a) know the configured name of the keyserver
b) resolve it to a set of intermediate names
c) resolve the intermediate names to a set of addresses
d) try to connect to those addresses, using the intermediate name of the server
for SNI and HTTP host.

But still, no PTR records are needed.

Okay, I get this error now. I had to implement a new option --disable-ipv4 to
make testing easier.

I have never seen the no permission message but only a general connection failed
error. I can try your suggestion of setting an explicit NoIPv6Traffic

We have several cases:

1. A pool accessed via round-robin A/AAAA record: We do not use the canonical hostname (i.e. from the PTR) but the name of the pool for the certificate. This is the classical way how keyserver pools.

2. A pool access via SRV records: The SRV record has the canonical name and thus we do not need a PTR lookup. But we need a address lookup.

3. A keyserver specified by its name: We alread have the name thus no need for PTR lookup.

4. A keyserver specified by literal IP address: We need a host name for the certificate. Either we take it from the PTR record or we reject TLS access. I don't think that is is a real world use case but for debugging it is/was really helpful. Should we reject hkps via literal IP addresses?

It is quite possible that some of these cases do not work right. I
have done only manual testing and the matrix is pretty complex: We
have all combinations of direct/Tor, v4 only, v6 only, v4, v6,
interface up, network down.

Right, by "duplicate host", I mean hosts reachable by several addresses
and in particular by v4 and v6. My test back when I originally
implemented the code showed that when hosts are down their other
addresses are also down. Without marking the host dead, the code
would have tried the same request on another address and would run
into the next timeout.

I also think that most delays are due to connection problems and not due to DNS
problems. And most connection problems are due to lost network access. There
we might need to tweak the code a bit similar to what I did for ADNS.

Here's a concrete example of how using PTR records gets things mixed up.

keyserver.stack.nl offers keyserver service on port 443.

It has an A record at 131.155.141.70.

But the ptr is to mud.stack.nl:

70.141.155.131.in-addr.arpa. 69674 IN PTR mud.stack.nl.

and the https SNI and HTTP Host: directives provide an entirely different
website depending on whether you access it with:

  https://mud.stack.nl/

or

  https://keyserver.stack.nl/

If you access it as https://hkps.pool.sks-keyservers.net/, you get the
"keyserver" view. But if you access it by the name in the PTR record
("mud.stack.nl") then you get the mud view (and a 404 on any /pks URLs)

Even more troubling is that dirmngr successfully connects to mud.stack.nl and
does the query, even though it is configured to only talk to
hkps.pool.sks-keyservers.net

This suggests that anyone able to spoof a PTR record to me can get my dirmngr to
send my potentially-sensitive keyserver queries to an entirely different webserver.

Should be fixed in the just released 2.1.18

I've moved the discussion about the need for PTR over to
T2928

In this ticket, let's focus on what happens when Tor has the NoIPv6Traffic flag
set. How should dirmngr respond in that case?

I think if it gets a "permission denied" from its tor socket (or from any proxy)
when it's trying to connect to a host, it should treat that host as dead and
move to try next one. If dirmngr knows that it is using tor, and it knows that
the address it is trying is also IPv6, it could also log a message about the
IPv6Traffic flag.

does that seem like the right set of changes needed?

if you add NoIPv6Traffic to your torrc, and restart tor, can you replicate the
problem?

To answer your question:

  Can you explain why dirmngr does the DNS roundtrip lookup, mapping
  from the pool's A and AAAA addresses back to names?  It seems like
  it'd be a lot simpler (and faster, and less error-prone) to avoid
  the PTR lookups if we have the IP addresses already.

If it is a plain server and not a pool, looking up the PTR is
necessary to get the hostname for SNI and possible also for Host:
header.

For a pool we would not need the name because the already known name
of the pool is used for SNI. However, to find duplicate hosts in the
hosttable it is useful to have the hostname. We also return the
actual used hostname to gpg for information purposes and to eventually
store this with the key as meta info.

Yes, we could the PTR lookup of pools faster or in the background -
but for now a simple appraoch is better for debugging.

FWIW, I am using Libassuan 2.4.3 plus one portability fix for BSDs.

And here is a log when using

  keyserver hkp://oteiza.siccegge.de

in dirmngr.conf (and also use-tor) of course:

DBG: chan_7 -> OK Dirmngr 2.1.18-beta67 at your service
connection from process 24314 (1000:1000)
DBG: chan_7 <- keyserver --resolve --hosttable
DBG: dns: libdns initialized (tor mode)
DBG: dns: getsrv(_pgpkey-http._tcp.oteiza.siccegge.de) -> 0 records
DBG: dns: libdns initialized (tor mode)
DBG: dns: resolve_dns_name(oteiza.siccegge.de): Success
resolve_dns_addr for 'oteiza.siccegge.de': 'oteiza.siccegge.de' [already known]
DBG: dns: resolve_dns_addr(): Success
DBG: chan_7 -> S # http://oteiza.siccegge.de:11371
DBG: chan_7 -> S # hosttable (idx, ipv6, ipv4, dead, name, time):
DBG: chan_7 -> S # 0 6 oteiza.siccegge.de
v6=[2a01:4a0:59:1000:223:9eff:fe00:100f]
DBG: chan_7 -> OK
DBG: chan_7 <- [eof]
handler for fd 7 terminated
handler for fd 7 started
DBG: chan_7 -> # Home: /home/wk/.gnupg
DBG: chan_7 -> # Config: /home/wk/.gnupg/dirmngr.conf
DBG: chan_7 -> OK Dirmngr 2.1.18-beta67 at your service
connection from process 24325 (1000:1000)
DBG: chan_7 <- GETINFO version
DBG: chan_7 -> D 2.1.18-beta67
DBG: chan_7 -> OK
DBG: chan_7 <- KS_GET -- 0xDEADBEEF
number of system provided CAs: 173
DBG: http.c:connect_server: trying name='oteiza.siccegge.de' port=11371
DBG: dns: resolve_dns_name(oteiza.siccegge.de): Success
DBG: http.c:1706:socket_new: object 0x00007f1420453df0 for fd 8 created
DBG: http.c:request:
DBG: >> GET /pks/lookup?op=get&options=mr&search=0xDEADBEEF HTTP/1.0\r\n
DBG: >> Host: oteiza.siccegge.de:11371\r\n
DBG: http.c:request-header:
DBG: >> \r\n
DBG: chan_7 -> S PROGRESS tick ? 0 0
DBG: chan_7 -> S SOURCE http://oteiza.siccegge.de:11371
DBG: (27779 bytes sent via D lines not shown)
DBG: chan_7 -> OK

I tried this with and without my local v6 interface up; both are
obviously the same. Bote that in both cases my resolver is on the
local network and accessed via v4 - but it should not matter because
due to use-tor 8.8.8.8 is used anyway.

Even with the Tor from testing I am stil having the IPv6Traffic flag
in my torrc - I am not sure whether this is still required.

Using a configuration with only "use-tor" and debug options, and no
keyserver nor certificates defined I used

  gpg-connect-agent --dirmngr 'keyserver --resolve --hosttable' /bye

several times until I got oteiza.siccegge.de as keyserver:

DBG: chan_7 <- keyserver --resolve --hosttable
DBG: chan_7 -> S # https://oteiza.siccegge.de:443
DBG: chan_7 -> S # hosttable (idx, ipv6, ipv4, dead, name, time):
DBG: chan_7 -> S # 0 hkps.pool.sks-keyservers.net
DBG: chan_7 -> S # . hkps.pool.sks-keyservers.net
DBG: chan_7 -> S # . --> 2 10 6 4 9 5 8 7 1* 3
DBG: chan_7 -> S # 1 6 4 oteiza.siccegge.de
v6=[2a01:4a0:59:1000:223:9eff:fe00:100f] v4=92.43.111.21
DBG: chan_7 -> S # 2 6 4 bone.digitalis.org v6=[2a00:14b0:4200:3000:27::27]
v4=212.12.48.27
DBG: chan_7 -> S # 3 6 prod00.keyserver.dca.witopia.net
v6=[2606:9500:201:1::141]
DBG: chan_7 -> S # 4 6 4 gpg.NebrWesleyan.edu v6=[2606:1c00:2802::b]
v4=192.94.109.73
DBG: chan_7 -> S # 5 6 4 hufu.ki.iif.hu v6=[2001:738:0:600:216:3eff:fe02:42]
v4=193.224.163.43
DBG: chan_7 -> S # 6 6 4 gozer.rediris.es v6=[2001:720:418:caf1::8]
v4=130.206.1.8
DBG: chan_7 -> S # 7 6 4 mud.stack.nl v6=[2001:610:1108:5011::70]
v4=131.155.141.70
DBG: chan_7 -> S # 8 4 ip-209-135-211-141.ragingwire.net v4=209.135.211.141
DBG: chan_7 -> S # 9 4 host-37-191-238-78.lynet.no v4=37.191.238.78
DBG: chan_7 -> S # 10 4 cryptonomicon.mit.edu v4=18.9.60.141
DBG: chan_7 -> OK

and then "gpg --recv-key deadbeef":

DBG: chan_7 <- KS_GET -- 0xDEADBEEF
DBG: http.c:connect_server: trying name='oteiza.siccegge.de' port=443
DBG: dns: resolve_dns_name(oteiza.siccegge.de): Success
DBG: http.c:1706:socket_new: object 0x00007f5d5000bea0 for fd 9 created
DBG: http.c:request:
DBG: >> GET /pks/lookup?op=get&options=mr&search=0xDEADBEEF HTTP/1.0\r\n
DBG: >> Host: hkps.pool.sks-keyservers.net:443\r\n
DBG: http.c:request-header:
DBG: >> \r\n
DBG: chan_7 -> S PROGRESS tick ? 0 0
DBG: chan_7 -> S SOURCE https://oteiza.siccegge.de:443
DBG: (27779 bytes sent via D lines not shown)

I did my test with tor 2.5.12-4 (jessie). I will ungrade to testing now and redo.

here's the example run from my modified reproducer script that focuses on
oteiza.siccegge.de:

gpg: keybox '/home/dkg/tmp/tmp.XgzSpI4Oy0/gpg/pubring.kbx' created
gpg: keyserver receive failed: Permission denied
S # hosttable (idx, ipv6, ipv4, dead, name, time):
S # 0 6 oteiza.siccegge.de v6=[2a01:4a0:59:1000:223:9eff:fe00:100f]
OK
2017-01-18 03:19:00 dirmngr[30881] listening on socket
'/home/dkg/tmp/tmp.XgzSpI4Oy0/gpg/S.dirmngr'
2017-01-18 03:19:00 dirmngr[30882.0] permanently loaded certificates: 0
2017-01-18 03:19:00 dirmngr[30882.0] runtime cached certificates: 0
2017-01-18 03:19:00 dirmngr[30882.0] failed to open cache dir file
'/home/dkg/tmp/tmp.XgzSpI4Oy0/gpg/crls.d/DIR.txt': No such file or directory
2017-01-18 03:19:00 dirmngr[30882.0] creating directory
'/home/dkg/tmp/tmp.XgzSpI4Oy0/gpg/crls.d'
2017-01-18 03:19:00 dirmngr[30882.0] new cache dir file
'/home/dkg/tmp/tmp.XgzSpI4Oy0/gpg/crls.d/DIR.txt' created
2017-01-18 03:19:01 dirmngr[30882.6] handler for fd 6 started
2017-01-18 03:19:01 dirmngr[30882.6] connection from process 30879 (1000:1000)
2017-01-18 03:19:01 dirmngr[30882.6] DBG: dns: libdns initialized (tor mode)
2017-01-18 03:19:02 dirmngr[30882.6] DBG: dns:
getsrv(_pgpkey-http._tcp.oteiza.siccegge.de) -> 0 records
2017-01-18 03:19:02 dirmngr[30882.6] DBG: dns: libdns initialized (tor mode)
2017-01-18 03:19:03 dirmngr[30882.6] DBG: dns:
resolve_dns_name(oteiza.siccegge.de): Success
2017-01-18 03:19:03 dirmngr[30882.6] resolve_dns_addr for 'oteiza.siccegge.de':
'oteiza.siccegge.de' [already known]
2017-01-18 03:19:03 dirmngr[30882.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 03:19:03 dirmngr[30882.6] number of system provided CAs: 142
2017-01-18 03:19:03 dirmngr[30882.6] DBG: http.c:connect_server: trying
name='oteiza.siccegge.de' port=11371
2017-01-18 03:19:05 dirmngr[30882.6] DBG: dns:
resolve_dns_name(oteiza.siccegge.de): Success
2017-01-18 03:19:05 dirmngr[30882.6] can't connect to 'oteiza.siccegge.de':
Permission denied
2017-01-18 03:19:05 dirmngr[30882.6] error connecting to
'http://oteiza.siccegge.de:11371': Permission denied
2017-01-18 03:19:05 dirmngr[30882.6] command 'KS_GET' failed: Permission denied
2017-01-18 03:19:05 dirmngr[30882.6] handler for fd 6 terminated
2017-01-18 03:19:05 dirmngr[30882.6] handler for fd 6 started
2017-01-18 03:19:05 dirmngr[30882.6] connection from process 30921 (1000:1000)
2017-01-18 03:19:05 dirmngr[30882.6] handler for fd 6 terminated

fwiw, i'm seeing fewer errors with this version than i was before, perhaps
because we're addressing servers via tor by name instead of by IP address, which
means that we're not tickling the IPv6 issue quite as often?

The failure with oteiza.siccegge.de might actually just be the IPv6 issue
itself, since there is no IPv4 address for it. I can actually force the issue
if i just add the following line to the dirmngr.conf generated in my reproducer
script:

    keyserver hkp://oteiza.siccegge.de

but of course it's a faster failure, because there isn't a dozen DNS A->PTR
round-trips.

Can you explain why dirmngr does the DNS roundtrip lookup, mapping from the
pool's A and AAAA addresses back to names? It seems like it'd be a lot simpler
(and faster, and less error-prone) to avoid the PTR lookups if we have the IP
addresses already.

I note here that the "oteiza.siccegge.de" domain name might be supplied by PTR
records for both its v4 and v6 addresses, and it appears to have a AAAA record,
but it doesn't have any *forward* A record.

I'm baffled by the fact that you're not seeing these errors, and not sure what
to do about it. What version of tor are you running? how is it configured?

i'm running the stock debian tor package, version 0.2.9.8-2.

I've tried with the latest patches and i still see failures :(

gpg: keybox '/home/dkg/tmp/tmp.nchsng7MNY/gpg/pubring.kbx' created
gpg: keyserver receive failed: Permission denied
S # hosttable (idx, ipv6, ipv4, dead, name, time):
S # 0 hkps.pool.sks-keyservers.net
S # . hkps.pool.sks-keyservers.net
S # . --> 9 11 6 4 10 5 7 8 2* 3 1
S # 1 6 4 sks.spodhuis.org v6=[2a02:898:31:0:48:4558:73:6b73] v4=94.142.242.225
S # 2 6 4 oteiza.siccegge.de v6=[2a01:4a0:59:1000:223:9eff:fe00:100f]
v4=92.43.111.21
S # 3 6 prod00.keyserver.dca.witopia.net v6=[2606:9500:201:1::141]
S # 4 6 4 gpg.NebrWesleyan.edu v6=[2606:1c00:2802::b] v4=192.94.109.73
S # 5 6 4 d hufu.ki.iif.hu v6=[2001:738:0:600:216:3eff:fe02:42]
v4=193.224.163.43 (1s)
S # 6 6 4 gozer.rediris.es v6=[2001:720:418:caf1::8] v4=130.206.1.8
S # 7 4 ip-209-135-211-141.ragingwire.net v4=209.135.211.141
S # 8 4 mud.stack.nl v4=131.155.141.70
S # 9 4 ams.sks.heypete.com v4=51.15.53.138
S # 10 4 host-37-191-238-78.lynet.no v4=37.191.238.78
S # 11 4 cryptonomicon.mit.edu v4=18.9.60.141
OK
2017-01-18 02:56:23 dirmngr[9098] listening on socket
'/home/dkg/tmp/tmp.nchsng7MNY/gpg/S.dirmngr'
2017-01-18 02:56:23 dirmngr[9099.0] permanently loaded certificates: 0
2017-01-18 02:56:23 dirmngr[9099.0] runtime cached certificates: 0
2017-01-18 02:56:23 dirmngr[9099.0] failed to open cache dir file
'/home/dkg/tmp/tmp.nchsng7MNY/gpg/crls.d/DIR.txt': No such file or directory
2017-01-18 02:56:23 dirmngr[9099.0] creating directory
'/home/dkg/tmp/tmp.nchsng7MNY/gpg/crls.d'
2017-01-18 02:56:23 dirmngr[9099.0] new cache dir file
'/home/dkg/tmp/tmp.nchsng7MNY/gpg/crls.d/DIR.txt' created
2017-01-18 02:56:24 dirmngr[9099.6] handler for fd 6 started
2017-01-18 02:56:24 dirmngr[9099.6] connection from process 9096 (1000:1000)
2017-01-18 02:56:24 dirmngr[9099.6] DBG: dns: libdns initialized (tor mode)
2017-01-18 02:56:25 dirmngr[9099.6] DBG: dns:
getsrv(_pgpkey-https._tcp.hkps.pool.sks-keyservers.net) -> 0 records
2017-01-18 02:56:25 dirmngr[9099.6] DBG: dns: libdns initialized (tor mode)
2017-01-18 02:56:27 dirmngr[9099.6] DBG: dns:
resolve_dns_name(hkps.pool.sks-keyservers.net): Success
2017-01-18 02:56:28 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:28 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'sks.spodhuis.org'
2017-01-18 02:56:28 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:29 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:29 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'oteiza.siccegge.de'
2017-01-18 02:56:29 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:29 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:29 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'prod00.keyserver.dca.witopia.net'
2017-01-18 02:56:29 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:30 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:30 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'gpg.NebrWesleyan.edu'
2017-01-18 02:56:30 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:31 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): No name
2017-01-18 02:56:31 dirmngr[9099.6] resolve_dns_addr failed while checking
'hkps.pool.sks-keyservers.net': No name
2017-01-18 02:56:32 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:32 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'hufu.ki.iif.hu'
2017-01-18 02:56:32 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:33 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:33 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'gozer.rediris.es'
2017-01-18 02:56:33 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:34 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Connection
closed in DNS
2017-01-18 02:56:34 dirmngr[9099.6] resolve_dns_addr failed while checking
'hkps.pool.sks-keyservers.net': Connection closed in DNS
2017-01-18 02:56:35 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:35 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'ip-209-135-211-141.ragingwire.net'
2017-01-18 02:56:35 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:36 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:36 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'hufu.ki.iif.hu' [already known]
2017-01-18 02:56:36 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:37 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:37 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'gpg.NebrWesleyan.edu' [already known]
2017-01-18 02:56:37 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:38 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:38 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'mud.stack.nl'
2017-01-18 02:56:38 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:38 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:38 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'gozer.rediris.es' [already known]
2017-01-18 02:56:38 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:39 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:39 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'sks.spodhuis.org' [already known]
2017-01-18 02:56:39 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:40 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:40 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'oteiza.siccegge.de' [already known]
2017-01-18 02:56:40 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:41 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:41 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'ams.sks.heypete.com'
2017-01-18 02:56:41 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:41 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:41 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'host-37-191-238-78.lynet.no'
2017-01-18 02:56:41 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:42 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:42 dirmngr[9099.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': 'cryptonomicon.mit.edu'
2017-01-18 02:56:42 dirmngr[9099.6] DBG: dns: resolve_dns_addr(): Success
2017-01-18 02:56:42 dirmngr[9099.6] DBG: http.c:connect_server: trying
name='hufu.ki.iif.hu' port=443
2017-01-18 02:56:45 dirmngr[9099.6] DBG: dns: resolve_dns_name(hufu.ki.iif.hu):
Connection closed in DNS
2017-01-18 02:56:45 dirmngr[9099.6] resolving 'hufu.ki.iif.hu' failed:
Connection closed in DNS
2017-01-18 02:56:45 dirmngr[9099.6] can't connect to 'hufu.ki.iif.hu': host not
found
2017-01-18 02:56:45 dirmngr[9099.6] error connecting to
'https://hufu.ki.iif.hu:443': Unknown host
2017-01-18 02:56:45 dirmngr[9099.6] marking host 'hufu.ki.iif.hu' as dead
2017-01-18 02:56:45 dirmngr[9099.6] DBG: http.c:connect_server: trying
name='oteiza.siccegge.de' port=443
2017-01-18 02:56:46 dirmngr[9099.6] DBG: dns:
resolve_dns_name(oteiza.siccegge.de): Success
2017-01-18 02:56:46 dirmngr[9099.6] can't connect to 'oteiza.siccegge.de':
Permission denied
2017-01-18 02:56:46 dirmngr[9099.6] error connecting to
'https://oteiza.siccegge.de:443': Permission denied
2017-01-18 02:56:46 dirmngr[9099.6] command 'KS_GET' failed: Permission denied
2017-01-18 02:56:46 dirmngr[9099.6] handler for fd 6 terminated
2017-01-18 02:56:46 dirmngr[9099.6] handler for fd 6 started
2017-01-18 02:56:46 dirmngr[9099.6] connection from process 9101 (1000:1000)
2017-01-18 02:56:46 dirmngr[9099.6] handler for fd 6 terminated

Thanks for the report. I can replicate this.

Thanks for the dumps. I was not able to get the permission denied errors. The
ENETDOWN errors on my site where due to a IPv6 not being enabled for Tor (during
most of my older tests I used the Torbrowser).

Anyway, With the patches from today, things are working much better now.
Please give it a try.

gpg: keybox '/home/dkg/tmp/tmp.0Ew9D45cz7/gpg/pubring.kbx' created
gpg: /home/dkg/tmp/tmp.0Ew9D45cz7/gpg/trustdb.gpg: trustdb created
gpg: key 7638D0442B90D010: public key "Debian Archive Automatic Signing Key
(8/jessie) <ftpmaster@debian.org>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1
S # hosttable (idx, ipv6, ipv4, dead, name, time):
S # 0 hkps.pool.sks-keyservers.net
S # . hkps.pool.sks-keyservers.net
S # . --> 15 14 13 12 11 10 19 18* 17 16 9 8 7 6 5 4 3 2 1
S # 1 6 [2a02:898:31:0:48:4558:73:6b73]
S # 2 6 [2a01:4a0:59:1000:223:9eff:fe00:100f]
S # 3 6 [2a00:14b0:4200:3000:27::27]
S # 4 6 [2606:9500:201:1::141]
S # 5 6 [2606:1c00:2802::b]
S # 6 6 [2001:bc8:4700:2300::10:f15]
S # 7 6 [2001:bc8:2515::1]
S # 8 6 [2001:720:418:caf1::8]
S # 9 6 [2001:470:1:116::6]
S # 10 4 216.66.15.2
S # 11 4 212.12.48.27
S # 12 4 209.135.211.141
S # 13 4 192.94.109.73
S # 14 4 163.172.29.20
S # 15 4 130.206.1.8
S # 16 4 94.142.242.225
S # 17 4 92.43.111.21
S # 18 4 51.15.53.138
S # 19 4 37.191.238.78
OK
2017-01-12 11:35:25 dirmngr[833] listening on socket
'/home/dkg/tmp/tmp.0Ew9D45cz7/gpg/S.dirmngr'
2017-01-12 11:35:25 dirmngr[834.0] permanently loaded certificates: 0
2017-01-12 11:35:25 dirmngr[834.0] runtime cached certificates: 0
2017-01-12 11:35:25 dirmngr[834.0] failed to open cache dir file
'/home/dkg/tmp/tmp.0Ew9D45cz7/gpg/crls.d/DIR.txt': No such file or directory
2017-01-12 11:35:25 dirmngr[834.0] creating directory
'/home/dkg/tmp/tmp.0Ew9D45cz7/gpg/crls.d'
2017-01-12 11:35:25 dirmngr[834.0] new cache dir file
'/home/dkg/tmp/tmp.0Ew9D45cz7/gpg/crls.d/DIR.txt' created
2017-01-12 11:35:26 dirmngr[834.6] handler for fd 6 started
2017-01-12 11:35:26 dirmngr[834.6] connection from process 831 (1000:1000)
2017-01-12 11:35:26 dirmngr[834.6] DBG: dns: libdns initialized (tor mode)
2017-01-12 11:35:27 dirmngr[834.6] DBG: dns:
getsrv(_pgpkey-https._tcp.hkps.pool.sks-keyservers.net) -> 0 records
2017-01-12 11:35:27 dirmngr[834.6] DBG: dns: libdns initialized (tor mode)
2017-01-12 11:35:28 dirmngr[834.6] DBG: dns:
resolve_dns_name(hkps.pool.sks-keyservers.net): Success
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2a02:898:31:0:48:4558:73:6b73]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2a01:4a0:59:1000:223:9eff:fe00:100f]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2a00:14b0:4200:3000:27::27]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2606:9500:201:1::141]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2606:1c00:2802::b]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:bc8:4700:2300::10:f15]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:bc8:2515::1]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:720:418:caf1::8]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:470:1:116::6]'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '216.66.15.2'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '212.12.48.27'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '209.135.211.141'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '192.94.109.73'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '163.172.29.20'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '130.206.1.8'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '94.142.242.225'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '92.43.111.21'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '51.15.53.138'
2017-01-12 11:35:28 dirmngr[834.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '37.191.238.78'
2017-01-12 11:35:28 dirmngr[834.6] DBG: http.c:connect_server: trying
name='51.15.53.138' port=443
2017-01-12 11:35:28 dirmngr[834.6] DBG: dns: resolve_dns_name(51.15.53.138): Success
2017-01-12 11:35:31 dirmngr[834.6] DBG: http.c:1706:socket_new: object
0x00007f57e400a5d0 for fd 8 created
2017-01-12 11:35:34 dirmngr[834.6] DBG: http.c:request:
2017-01-12 11:35:34 dirmngr[834.6] DBG: >> GET
/pks/lookup?op=get&options=mr&search=0x126C0D24BD8A2942CC7DF8AC7638D0442B90D010
HTTP/1.0\r\n
2017-01-12 11:35:34 dirmngr[834.6] DBG: >> Host:
hkps.pool.sks-keyservers.net:443\r\n
2017-01-12 11:35:34 dirmngr[834.6] DBG: http.c:request-header:
2017-01-12 11:35:34 dirmngr[834.6] DBG: >> \r\n
2017-01-12 11:35:37 dirmngr[834.6] handler for fd 6 terminated
2017-01-12 11:35:37 dirmngr[834.6] handler for fd 6 started
2017-01-12 11:35:37 dirmngr[834.6] connection from process 841 (1000:1000)
2017-01-12 11:35:37 dirmngr[834.6] handler for fd 6 terminated

gpg: keybox '/home/dkg/tmp/tmp.swbfPRERsO/gpg/pubring.kbx' created
gpg: keyserver receive failed: Server indicated a failure
S # hosttable (idx, ipv6, ipv4, dead, name, time):
S # 0 hkps.pool.sks-keyservers.net
OK
2017-01-12 11:36:01 dirmngr[851] listening on socket
'/home/dkg/tmp/tmp.swbfPRERsO/gpg/S.dirmngr'
2017-01-12 11:36:01 dirmngr[852.0] permanently loaded certificates: 0
2017-01-12 11:36:01 dirmngr[852.0] runtime cached certificates: 0
2017-01-12 11:36:01 dirmngr[852.0] failed to open cache dir file
'/home/dkg/tmp/tmp.swbfPRERsO/gpg/crls.d/DIR.txt': No such file or directory
2017-01-12 11:36:01 dirmngr[852.0] creating directory
'/home/dkg/tmp/tmp.swbfPRERsO/gpg/crls.d'
2017-01-12 11:36:01 dirmngr[852.0] new cache dir file
'/home/dkg/tmp/tmp.swbfPRERsO/gpg/crls.d/DIR.txt' created
2017-01-12 11:36:02 dirmngr[852.6] handler for fd 6 started
2017-01-12 11:36:02 dirmngr[852.6] connection from process 849 (1000:1000)
2017-01-12 11:36:02 dirmngr[852.6] DBG: dns: libdns initialized (tor mode)
2017-01-12 11:36:12 dirmngr[852.6] DBG: dns:
getsrv(_pgpkey-https._tcp.hkps.pool.sks-keyservers.net): Server indicated a failure
2017-01-12 11:36:12 dirmngr[852.6] command 'KS_GET' failed: Server indicated a
failure <Unspecified source>
2017-01-12 11:36:12 dirmngr[852.6] handler for fd 6 terminated
2017-01-12 11:36:12 dirmngr[852.6] handler for fd 6 started
2017-01-12 11:36:12 dirmngr[852.6] connection from process 854 (1000:1000)
2017-01-12 11:36:12 dirmngr[852.6] handler for fd 6 terminated

gpg: keybox '/home/dkg/tmp/tmp.vOaRFt7s4L/gpg/pubring.kbx' created
gpg: keyserver receive failed: Permission denied
S # hosttable (idx, ipv6, ipv4, dead, name, time):
S # 0 hkps.pool.sks-keyservers.net
S # . hkps.pool.sks-keyservers.net
S # . --> 15 14 13 12 11 10 19 18 17 16 9 8 7 6 5 4 3 2* 1
S # 1 6 [2a02:898:31:0:48:4558:73:6b73]
S # 2 6 [2a01:4a0:59:1000:223:9eff:fe00:100f]
S # 3 6 [2a00:14b0:4200:3000:27::27]
S # 4 6 [2606:9500:201:1::141]
S # 5 6 [2606:1c00:2802::b]
S # 6 6 [2001:bc8:4700:2300::10:f15]
S # 7 6 [2001:bc8:2515::1]
S # 8 6 [2001:720:418:caf1::8]
S # 9 6 [2001:470:1:116::6]
S # 10 4 216.66.15.2
S # 11 4 212.12.48.27
S # 12 4 209.135.211.141
S # 13 4 192.94.109.73
S # 14 4 163.172.29.20
S # 15 4 130.206.1.8
S # 16 4 94.142.242.225
S # 17 4 92.43.111.21
S # 18 4 51.15.53.138
S # 19 4 37.191.238.78
OK
2017-01-12 11:36:23 dirmngr[866] listening on socket
'/home/dkg/tmp/tmp.vOaRFt7s4L/gpg/S.dirmngr'
2017-01-12 11:36:23 dirmngr[867.0] permanently loaded certificates: 0
2017-01-12 11:36:23 dirmngr[867.0] runtime cached certificates: 0
2017-01-12 11:36:23 dirmngr[867.0] failed to open cache dir file
'/home/dkg/tmp/tmp.vOaRFt7s4L/gpg/crls.d/DIR.txt': No such file or directory
2017-01-12 11:36:23 dirmngr[867.0] creating directory
'/home/dkg/tmp/tmp.vOaRFt7s4L/gpg/crls.d'
2017-01-12 11:36:23 dirmngr[867.0] new cache dir file
'/home/dkg/tmp/tmp.vOaRFt7s4L/gpg/crls.d/DIR.txt' created
2017-01-12 11:36:24 dirmngr[867.6] handler for fd 6 started
2017-01-12 11:36:24 dirmngr[867.6] connection from process 864 (1000:1000)
2017-01-12 11:36:24 dirmngr[867.6] DBG: dns: libdns initialized (tor mode)
2017-01-12 11:36:26 dirmngr[867.6] DBG: dns:
getsrv(_pgpkey-https._tcp.hkps.pool.sks-keyservers.net) -> 0 records
2017-01-12 11:36:26 dirmngr[867.6] DBG: dns: libdns initialized (tor mode)
2017-01-12 11:36:27 dirmngr[867.6] DBG: dns:
resolve_dns_name(hkps.pool.sks-keyservers.net): Success
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2a02:898:31:0:48:4558:73:6b73]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2a01:4a0:59:1000:223:9eff:fe00:100f]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2a00:14b0:4200:3000:27::27]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2606:9500:201:1::141]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2606:1c00:2802::b]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:bc8:4700:2300::10:f15]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:bc8:2515::1]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:720:418:caf1::8]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '[2001:470:1:116::6]'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '216.66.15.2'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '212.12.48.27'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '209.135.211.141'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '192.94.109.73'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '163.172.29.20'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '130.206.1.8'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '94.142.242.225'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '92.43.111.21'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '51.15.53.138'
2017-01-12 11:36:27 dirmngr[867.6] resolve_dns_addr for
'hkps.pool.sks-keyservers.net': '37.191.238.78'
2017-01-12 11:36:27 dirmngr[867.6] DBG: http.c:connect_server: trying
name='2a01:4a0:59:1000:223:9eff:fe00:100f' port=443
2017-01-12 11:36:27 dirmngr[867.6] DBG: dns:
resolve_dns_name(2a01:4a0:59:1000:223:9eff:fe00:100f): Success
2017-01-12 11:36:27 dirmngr[867.6] can't connect to
'2a01:4a0:59:1000:223:9eff:fe00:100f': Permission denied
2017-01-12 11:36:27 dirmngr[867.6] error connecting to
'https://[2a01:4a0:59:1000:223:9eff:fe00:100f]:443': Permission denied
2017-01-12 11:36:27 dirmngr[867.6] command 'KS_GET' failed: Permission denied
2017-01-12 11:36:27 dirmngr[867.6] handler for fd 6 terminated
2017-01-12 11:36:27 dirmngr[867.6] handler for fd 6 started
2017-01-12 11:36:27 dirmngr[867.6] connection from process 869 (1000:1000)
2017-01-12 11:36:27 dirmngr[867.6] handler for fd 6 terminated

Here's the reproducer script i'm using:

--------

#!/bin/bash

WORKDIR=$(mktemp -d)
export GNUPGHOME="$WORKDIR/gpg"
mkdir -p -m 0700 "$GNUPGHOME"
cat > "$GNUPGHOME/dirmngr.conf" <<EOF
debug dns,network
verbose
use-tor
log-file $WORKDIR/dirmngr.log
EOF

gpg --recv 126C0D24BD8A2942CC7DF8AC7638D0442B90D010

gpg-connect-agent --dirmngr 'keyserver --hosttable' /bye

cat "$WORKDIR/dirmngr.log"

rm -rf "$WORKDIR"

I just ran it three times in a row, and i got three different results, which
i'll paste as separate messages for easier visibility.

Can you run dirmngr with

debug dns,network
verbose

I don't think that gnutls debug is required.

They don't solve the bug for me, unfortunately. with those patches applied, i
now get "permission denied" errors:

an 11 15:57:18 alice dirmngr[20203]: DBG: gnutls:L3: ASSERT:
mpi.c[_gnutls_x509_read_uint]:246
Jan 11 15:57:18 alice dirmngr[20203]: DBG: gnutls:L5: REC[0x7f07c0008640]:
Allocating epoch #0
Jan 11 15:57:18 alice dirmngr[20203]: can't connect to
'2a02:898:31:0:48:4558:73:6b73': Permission denied
Jan 11 15:57:18 alice dirmngr[20203]: error connecting to
'https://[2a02:898:31:0:48:4558:73:6b73]:443': Permission denied

which also don't mark the IPv6 address as dead, so they're effectively permanent
until i clear them out.

As a workaround, i've been clearing out all IPv6 addresses with this terrible hack:

0 dkg@alice:~$ cat bin/dirmngr-flush-ipv6
#!/bin/bash

drop all IPv6 keyservers from dirmngr:

gpg-connect-agent --dirmngr 'keyserver --hosttable' /bye |\

awk '/\[.*:.*\]/{ print "keyserver --dead " $5 } ' |\
gpg-connect-agent --dirmngr

0 dkg@alice:~$

I just pushed two patches:
76fb2fe dirmngr: Mark hosts dead on ENETDOWN.
09aeac4 dirmngr: Fix Tor access for v6 addresses.
which should solve this bug.

The reason why we see an EINVAL for a v6 address is that
we create the socket for AF_INET6 but then use that very socket with AF_INET to
sonnect to the socks5 proxy. Cleary a socket for v6 can't be used to connect to
a v4 address.

Please try current master where I hopefully fixed T2451. This may also fix
this issue.

Fixed in master:

0cc975d dirmngr: Use "pgpkey-hkps" and "pgpkey-hkp" for SRV record lookups.
c2cbe2f dirmngr: Do not use a SRV record for HKP if a port was specified.

This seems to be closely related to T2451.

The ML discussion started during my fall vacation and thus scrolled out of my
sight :-(. While doing some unrelated SRV experiments yesterday, I figured the
problem myself and K_F pointed me to the discussion.

The actual cause for the bugs might be that I missed to forward-port David's
patch from 2009 for changing the service name to the 2.1 branch. I will further
investigate. The move to libdns introduced other bugs and covered the actual bug.

We should use the wrapper as long as we don't have full control over libldap
(select, timeout, etc.)

I was wrong about Tor and IPv6 -- Tor has had IPv6 support for years, so
something else is wrong...

On Thu, Dec 1, 2016 at 10:33 AM, Werner Koch via BTS <gnupg@bugs.g10code.com

wrote:

Can you please switch to 2.1.16? We fixed the inotify based socket removal
detection (issue2756).

On Wed, Nov 30, 2016 at 10:28 AM, Werner Koch via BTS <
gnupg@bugs.g10code.com> wrote:

If you are on Linux you may simply remove the socket files and the daemons will
terminate.

On Tue, Nov 29, 2016 at 7:14 PM, Werner Koch via BTS <gnupg@bugs.g10code.com>
wrote:

[...]
thanks for the clarifications, I was 95% there but that rounds it up to the full
picture.

   GNUPGHOME=$(pwd) gpg-agent --daemon -v MYSCRIPT

This starts the agents and runs MYSCRIPT (/bin/sh is nice for testing). After
MYSCRIPT terminates the agent will be terminated within the next minute.

We more or less do that already, and I can confirm it does "terminate the next
minute".
Unfortunately "the next minute" isn't soon enough for my umount.

So I thought for now I might have to rely on an extra call of "gpgconf --kill
dirmngr" to be sure. But I have found that this doesn't end it for me - maybe
related to our custom GNUPGHOME?
It doesn't seem to stop as intended via gpgconf: (neither with nor without
GNUPGHOME):

$ ps axlf | egrep 'gpg|dirmngr'
1 0 18021 1 20 0 176840 6236 - Ssl ? 0:00 dirmngr --
daemon --homedir /tmp/tmped9zl1ip
$ gpgconf --kill dirmngr
$ ps axlf | egrep 'gpg|dirmngr'
1 0 18021 1 20 0 176840 6236 - Ssl ? 0:00 dirmngr --
daemon --homedir /tmp/tmped9zl1ip
$ GNUPGHOME="/tmp/tmped9zl1ip" gpgconf --kill dirmngr
$ ps axlf | egrep 'gpg|dirmngr'
1 0 18021 1 20 0 176840 6236 -
(Also did the same as root without effect, but avoid filling this report only
with logs)

The more obvious but less graceful "sudo killall dirmngr |:" cleans it up.

The reason why your bind mount fails are likely the socket files created there.

via lsof I only found the following to be related (the /dev bind umount is what
fails).
dirmngr 6771 root 1r CHR LG,0x80000 1,9 0t0
11 /tmp/tmptmucmfm0/target/dev/urandom
dirmngr 6771 root 2w CHR W,LG 1,3 0t0
6 /tmp/tmptmucmfm0/target/dev/null
gpg-agent 6776 root 0r CHR LG 1,3 0t0
6 /tmp/tmptmucmfm0/target/dev/null
gpg-agent 6776 root 1w CHR W,LG 1,3 0t0
6 /tmp/tmptmucmfm0/target/dev/null
gpg-agent 6776 root 2w CHR W,LG 1,3 0t0
6 /tmp/tmptmucmfm0/target/dev/null

Not sure, but I'd not expect these are the sockets.
My assumption would be the open /dev/urandom - especially as I found that I only
need to get rid of dirmngr to proceed.

The solution to this are //run/user based sockets. If you create a directory
/run/user/UID gnupg will create a gnupg sub-directory and place the socket
files there.

gpg-conf --list-dirs

can be used to check that.

I created a log based on that check and lsof (the view from inside the chroot):
gpgconf --list-dirs
lsof +fg /dev
gpgconf --kill dirmngr
lsof +fg /dev
pkill gpg-agent
pkill dirmngr
lsof +fg /dev

I'll attach the full log with this post.
You can find there that the agent and dirmngr do not go away on the gpgconf --
kill.
The socket path OTOH looks ok with "agent-socket:/root/.gnupg/S.gpg-agent"

I added this as first command in chroot to give it a try at least:
$ mkdir -p /run/user/$(id -u)
It "worked" in the sense that the dirs changed e.g. from:
dirmngr-socket:/root/.gnupg/S.dirmngr
to
dirmngr-socket:/run/user/0/gnupg/S.dirmngr
But as I expected that did not change the behavior of blocking to umount the /dev

I really think it is not the sockets but e.g. the /dev/urandom it keeps open.

Is there any way to get the agent&dirmngr from "away next minute" to be "away
before exiting the calling command"?

BTW - I wondered if it would be enough if gpg-agent can stay and to only kill
dirmngr - but it seems I need both gone to let me umount.

While looking at the problem I found a corner case related to a shutdown and
fixed that.

I also tried to close the listening socket after the first shutdown event. I
reverted that because the effect is that a client trying to connect immediately
gets a failure and then starts a new dirmngr - which is not the idea of a shutdown.

The man pages notes:

SIGTERM

   Shuts down the process but waits until all current requests are
   fulfilled.  If the process has received 3  of these signals
   and requests are still pending, a shutdown is forced.  You may
   also use
         gpgconf --kill dirmngr
   instead of this signal

thus this is by design and identical to what gpg-agent does. IIRC, there was a
regression for some time, fixed in 2.1.16. So this fixed regression is what you
see as a bug.

However, the process should not anymore listen for new connections as soon as a
shutdown is pending. That needs to be fixed.

The agent is not directly involved in keyserver lookups. Instead the dirmngr is
spawned and does this for you. In fact, all network access goes through dirmngr.
However the agent is a mandatory part of gnupg-2 - all private keys are
processed solely by the agent. Thus removing the agent is not an option.
In your case the agent is launched to check whether a private key is available
for an imported public key.

You can stop the agent or the dirmngr using "gpgconf --kill dirmngr"
("gpg-agent"). For your use case a better solution will be to run your script
under the control of the agent (ie. explicily starting the agent):

  GNUPGHOME=$(pwd) gpg-agent --daemon -v MYSCRIPT

This starts the agents and runs MYSCRIPT (/bin/sh is nice for testing). After
MYSCRIPT terminates the agent will be terminated within the next minute.

The reason why your bind mount fails are likely the socket files created there.
I had the same problem with my g13 based disk encryption. The solution to this
are //run/user based sockets. If you create a directory /run/user/UID gnupg
will create a gnupg sub-directory and place the socket files there.

gpg-conf --list-dirs

can be used to check that.

Patch 0001 should be applied to 2.3

Please do not use "checking-upstream-swdb" patch.

Sure, for Debian and other distros the version number is of no use and should
not be used (I am still annoyed by xlockscreen thing). However disabling this
in dirmngr is the wrong approach. It should be disabled in tools which actually
use that service (e.g. KMail). The SWDB file carries more version information
than just GPA and is thus useful for developers who build their own version of
GPA or their own Windows installer. It has also nothing to do with the wakeups.

Having a dirmngr installed which does not work as described is a bad idea.

BTW: although we won't be able to implement key retrieval queueing into dirmngr
(e.g. for use with --auto-key-retrieve) in time for the Debain freeze, we will
add this later so that it may be available in a later point release. Obviously
this needs regualr wakeups to test for network connectivity and to process the
queue.

I just pushed the LDAP reaper patch for 2.1.17.

The LDAP stuff is mainly used for CRLs and is often hard to deploy because often
proxies are needed etc. I don't know a public one which is reliable enough for
testing. The one I used mostly was related to certain smartcards but those
cards expire faster than software can be deployed. Fortunately most public CRLs
are available via HTTP.

Another use are LDAP keyservers. I do not know a public service, Some
keyserver operators run them privately and Ireply on them to test GnUPG's support.

I've updated the patch series here to the series we're using in debian for 2.1.16.

D389: 918_0005-dirmngr-Drop-useless-housekeeping.patch