@gouttegd good catch!

I just assumed that is an ntbtls problem.

If I understand correctly, this is exactly the same problem that the one we encountered some time ago in the code dealing with fetching keys from HTTP (--fetch-keys), and that we fixed with this patch.

fwiw, the bug looks like it's in send_request in ks-engine-hkp.c, which re-uses the http_session object without re-initializing its tls_session member.

thanks for the triage, @werner!

We need --keyserver in gpg for just one reason: backward compatibility.

thanks for fixing that error message, @werner. As @Valodim points out in discusson about hagrid, a gpg.conf keyserver option (deprecated according to the documentation) overrides the dirmngr.conf keyserver option (not deprecated according to the documentation.

We only supported SHA-1 signed OCSP requests. Fix will go into 2.2.16.

I doubt that we are going to implement this.

Interesting tinge: The main CRL of the dgn.de CA uses a nextUpdate in the year 2034 (15 years in the future) which would force dirmngr to cache the CRL until then. However, the CRL of the intermediate certificate has a nextUpdate only one month in the future. There is currently no entry in that second level CRL, so their idea might be that an updated second level CRL will also trigger a reload of the main CRL. I have not checked how we implement that in Dirmngr but I doubt that such a thing will work for us and that it is in any way standard compliant.

Are you not reading what I am saying to you?? Once again, your explanation is INVALID because that would mean that gnupg would be BROKEN, because it would be a NON-COMPLIANT http client according to the RFC I quoted.

I explained why the keyserver access requires access to the DNS. If that is not possible the keyserver code will not work. If you don't allow DNS to work you either have to use Tor (which we use to also tunnel DNS requests) or get your keys from elsewhere. Also note that the keyserver network is current several broken and under DoS and thus it is unlikely that it can be operated in the future.

I agree with @dkg here.

"requires too much changes" i can understand.

This requires too much changes and does not reflect the reality. It actually makes debugging harder for us.

Thanks

I think you are saying that dirmngr receives the query term as escaped data in the assuan connection from the dirmngr client (typically, gpg, which itself decides how to percent-escape what it feeds into libassuan).

This is easy to explain: dirmngr receives already escaped data and that is what you see in the log. For proper parsing of the URI the escaping needs to be removed and only before sending the request the required escaping is applied. '@', '<', and '>' do not need to be escaped and thus you see them as they are.

I removed this specialized error message. Thanks for reporting.

This is particularly bad for users who have manually specified a given keyserver in dirmngr.conf, because even a transient failure in that keyserver will prevent them from any future keyserver requests until dirmngr decides that the "death" has worn off.

further testing suggests that the invalid URI issue is only present for dirmngr's --keyserver option, and gpg's deprecated --keyserver option actually accepts schema-less hostnames.

see also T4467

I just noticed that dirmngr(8)'s documentation for its --keyserver option says:

Note that even sending a HUP to dirmngr, when it is in this autodetection mode that observed tor at the start, is insufficient to have it re-run the autodetection. You have to explicitly terminate dirmngr to get it to unlearn the autodetected presence of Tor. This is subtly hinted at in dirmngr(8), but no justification is given for it.

One of the things that dirmngr has going for it is that it tracks the current network state, and it would be nice to be able to reuse that state across sessions. If an ephemeral keyring can't use a shared dirmngr, there are fewer arguments for having dirmngr in the first place, and people might be more justified in replacing it with things like https://gitlab.com/anarcat/scripts/blob/master/openpgp-key-get

I don't anymore think this is a high priority request. BTW, A more real problem than several dirmngr instances is multi-user access to smartcards.

HTTP/1.1 spec, RFC 7230, Section 5.4, paragraph 2:
https://tools.ietf.org/html/rfc7230#section-5.4

Please be so kind and point me to the specs stating that you should put the IP address into Host:

It's up to GPG to send the Host header that shows the user's intent.

So in short you want:

1. Allow to specify a keyserver by IP without any DNS lookups.
2. When connecting via IP use the IP address for Host:.

gpg4win 3.1.6 is released which contains this fix.

Also might I add, this used to work perfectly fine in gnupg14. It seems that somewhere along the line a regression was introduced that is causing this erroneous non-compliant behavior in the HTTP client.

Why? Your explanation is invalid because it implicates dirmngr's HTTP client as not comforming to the spec laid out by the RFC. I've quite clearly explained--and backed up with the spec itself--that when a proxy variable is configured, a client should not be doing DNS lookup of the destination hostname. Is there something about that you are not understanding?

Please show an example regarding something else than a failed access to a pool of keyservers. I explained why it can't work for pools for you.

Yes you can, and no you do not. Don't believe me? Then read the spec. At no point does the spec say that there is "nothing that can be done" when a hostname cannot be resolved when connecting through a proxy. In fact, it states precisely the opposite, describing the exact procedure a client should take when making a request through a proxy. See section 5.3, paragraph 3:

No we can't we need to know the IP addresses to handle the pools. I have given a workaround for you in my previous comment. You can also use install Tor which we can use for DNS resolving.

There is a solution for it:

Btw. I only noticed this now as I always had "disable-tor" in my config but recently removed it for testing.

So, the keyserver operator had thrown in a hockeypuck server in the pool, causing this.. While the keyserver remains in the exclude list until confirmation it has been resolved, that explains the behavior and it has been made clear that separate software needs to use different names in the future.

@kristianf we talked about this on Saturday evening. Would you be so kind and have a quick look at the problem with the hu server?

Hi Werner and thanks for looking into this.

According to sks-keyservers.net both servers you mention run the very same software. Thus I would like to understand why you think they require the use of a legacy option.

I want to have this fixed for the next release so prio high.

Oops. Assignee removal was an accident. Sorry for the noise here ;-)

Just as a note: To workaround this you can also place "no-use-tor" into %APPDATA%\gnupg\dirmngr.conf (you might need to create that file) %APPDATA% expands to something like "c:\users\yourname\appdata\roaming"

In T3381#121973, @madhon wrote:

In T3381#121972, @Spiker wrote:

On Win 10 Pro it looks like File Transfer Server.exe is running on port 9050 which could be causing the issue. See screenshots.

That process is the one i killed which is part of Asus Wi-Fi Go

In T3381#121972, @Spiker wrote:

On Win 10 Pro it looks like File Transfer Server.exe is running on port 9050 which could be causing the issue. See screenshots.

On Win 10 Pro it looks like File Transfer Server.exe is running on port 9050 which could be causing the issue. See screenshots.

Apparently i had a ASUS Wi-Fi go process listening on that port (even though i thought had uninstalled it), killing the process also allows dirmngr to start

Thanks you very much for your help! I think we have it. \o/

Running with the --no-use-tor results in output ending with OK Dirmngr 2.2.11 at your service, attached is the procmon output , to clear up one thing q4master.idsoftware.com points to 127.0.0.1 in my hosts file (in addition to localhost also pointing to 127.0.0.1), but it seems the issue is with the tor check

I see some strangeness:
A TCP Connect: q4master.idsoftware.com:4862 -> q4master.idsoftware.com:9050
and TCP Send: q4master.idsoftware.com:4862 -> q4master.idsoftware.com:9050

Done, See attached

I'm thinking of how to move this forward.
The problem is that we (the developers) can't reproduce this at all and the debug output does not show anything.

Has anybody discovered a fix for this issue? I'm running Win 10 Pro with Gpg4win v3.1.5. Dirmngr is still not executing and just hangs.

So if your DNS resolver does not tell us the IP addresses, we can't do anything about it.

If you specify a pool of keyservers dirmngr selects a keyserver on its won from the pool. This is so that it can use its own heuristics to detect whether a keyserver is dead and then retry another one. Now the default is a pool and your specified keyserver.ubuntu.com is also a pool (of two servers). So if your DNS resolver does not tell us the IP addresses, we can't do anything about it.

Will go into 2.1.12 to be released next week.