I agree that this is a tricky problem, but it should really be improved.

The problem is not to check whether there is a connection but on how to decide whether something is a pool or an explictly added single keyserver and how often should we try to connect or read from it. Without marking hosts as dead the auto search features won't work well.

@Valodim probably not so much as dirmngr might behave differently and not mark hosts as dead.

The proper solution is of course to use pkill instead of killall. SCNR.

I can attest to the "growing bit of popular lore": Roughly half the support requests I get to support@keys.openpgp.org boil down to an exchange of "it just doesn't work with a 'general error' message" -> "try killall dirmngr" -> "that did it". I have heard similar stories from @patrick from Enigmail users, and more than once heard people applying poweruser trickery like "I just have killall dirmngr in my resume.d".

The LDAP code is actually in very bad shape because @neal added it without utilizing the ldap wrapper and thus a timeout won't work reliable.

Unusable v6 interfaces are now detected on Windows and then not used.

The manual states that --standard-resolver is mostly for debugging. The reason you get an "not enabled" is that we can't allow direct DNS queries in Tor mode which would happen with the system (standard) DNS resolver.

See also D475.

Ping.

There is a growing bit of popular lore in the GnuPG community that "when keyserver operations fail, you solve that problem with killall dirmngr." I believe this suggestion is potentially damaging (the long-running daemon may be in the middle of operations for a client that you don't know about), but i suspect it is circulating as advice because it resolves the situation outlined in this ticket. For whatever ephemeral reason, dirmngr gets stuck, and fails to notice that this situation has resolved itself.

GnuPG ships a non-PKI certificate, specifically to authenticate hkps.pool.sks-keyservers.net. Now due to an implementation detail, this has been shown to potentially lead to authentication of other domains by this certificate, if a maintainer changes the default keyserver via the DIRMNGR_DEFAULT_KEYSERVER variable in configure.ac. Now arguably, this variable isn't exposed via ./configure, so it's not "officially" configurable - but evidently maintainers do want to change it. A trivial one-line patch was supplied to change the unintended and potentially security-problematic behavior into the (I believe) obviously intended one.

$ gpg-connect-agent --dirmngr 'getinfo version' /bye
D 2.2.17
OK

Can you check which dirmngr version you are running

gpg-connect-agent --dirmngr 'getinfo version' /bye

thanks for the dns explanation - IMHO, there should be added something about that in the wiki
When it does not work for you on http1 either, then I guess, it's really just some outdatedness of my gpg/dirmngr and this ticket can be closed.

It does not work either. Your problem is the use of a wildcard DNS for archlinux32.org:

The test above was with gpg master but I got the same result with current 2.2:

ok, I disabled it again. btw: why do we need openpgpkey.archlinux32.org in the cert? Is this standard or did I misconfigure something?

Thanks. Here is a dirmngr log:

I set archlinux32.org back to http2 - so you can see for yourself, how gpg fails to retrieve the key for buildmaster@archlinux32.org

I believe, it means, that it may fall back to http1.1 - the documentation is not clear to me on this.
A simple test however shows, that at least curl has no problems to use http1.1 or http1.0 with the http2 enabled nginx.

Does your ngix configuration mean that there is no fallback to standard http?

In T2300#90092, @aheinecke wrote:

Ah nevermind. I think myself that this is nobug and current behavior is correct.

To implement / test the "not literally RFC compliant but in practice better" behavior let us call this now a wish and feature request as there are certificates in the wild other then intevation's and customers in large institutions run into that.

Will be in 2.2.18

WKD and DANE/OPENPGPKEY offer rather distinct properties. I'd be hard-pressed to say that one is "better" than the other without understanding the threat model and concerns of the evaluator:

DNSSEC is a centralized CA system. Just different than the TLS one. Given that Certificate Transparency exists I'd say DNSSEC is less transparent than TLS. For example if you happen to have a .ly domain then the Libyan can silently control your signed zone. Given that there is no CT for DNSSEC they can do so selectively, for any connection they want. It wouldn't be the first problem with them.

In T4618#128103, @wiktor-k wrote:

I'm left wondering: are there cases where OPENPGPKEY would be preferred over WKD?

Just a note that we're now shipping this patch in debian unstable. It would be great if it was merged upstream.

I see. I am also mostly testing with ntbtls so I was wondering about the report. Thanks for reporting and fixing.

While I understand incorrectness, the risk in practice is not that high. So, I put this as "normal" priority.

Pushed the change to master as well as 2.2 branch.

Is this really necessary to duplicate functionality that already is provided by Web Key Directory?

With NTBTLS, it seems it works correctly.

I agree, many currently-shipped DNS client library implementations do not provide DNSSEC validity checks.

Sure it is not validated. Standard clients do not provide the system features to do that. That is one of the problems with DNSSEC adoption - it works only for servers in practice.

Ah, that makes sense, good catch. Seems this is just an issue of documentation, then.

And of course, thanks for your fix.

Applied to both branches. I have run no tests myself, though.

Fix will be in 2.2.17

I tried to implement this but this is troublesome for other programs using the interface because a common patter is to use --search-keys to get a listing and then use --recv-key to import the keys - That won't work and will require changes to --recv-key too. Thus this change will not go into 2.2. Anyway, it is not dangerous to have --search-keys because the new default for import from keyservers will be to strip all key-signatures.

I asked you to carry this to a mailing list and not re-open this task.

My plan is to let --search-key be the same as locate-key but without local lookups, thus it will be the same as

That was pretty easy to reproduce thanks to your still not working server.

I did some manual tests using netcat and KS_FETCH to test the redirection.

I think you're suggesting accepting *any* path if the hostname of the proposed redirection matches openpgpkey.example.org when querying the WKD direct URL for an @example.org address. That would also be a fine solution from my point of view.

I head the same idea when I read your configuration. Given that the advanced lookup was not reallydeployed (see T4590) I also expect that we will receive complains now that it works. Thus white listing any "openpgpkey." seems to me a reasonable easy solution.

Will be in 2.2.17

Oh dear, that happens if one is always on master. I simply forgot to cherry pick the change from master back in November.
Two commits, though.

@werner, thanks for the pointer to the report, that's certainly useful. And i'm happy that organizations like SektionEins are doing GnuPG audits and publishing their results regardless of who paid for them.

See https://sektioneins.de/en/blog/18-11-23-gnupg-wkd.html for details. In short they fear that companies using IP based security for internal services can be attacked via redirect request and in particular becuase that can happen in the background without the user noticing. I am not concerned but we had long lasting discussions also with protonmail about this and the result was that we need to have this protection. We do not know who requested and paid for the audit from SektionEins and they won't tell us.

Thanks for the pointer, @werner. Certainly we want T4590 fixed.

We need to rewrite the Location to avoid a CSRF attack. See fa1b1eaa4241ff3f0634c8bdf8591cbc7c464144

I should add that i don't really care whose fault it is if the software is broken by some downstream. if it harms any users, and we can fix it, we should fix it, especially if the fix is easy.

We're writing free software, which we know that people use and modify downstream. if we know that the software has a particular sharp edge that people who are modifying it are likely to cut themselves on, we have two options:

Come on, if someone changes the software and breaks it, it is their's fault ant not ours. The whole thing on which keyserver and certificate to use as been discussed ad nausea in the past. Given all the problems with the keyservers I do not see a reason to change it right away to a state we had before. Keyserver code is pretty hard to test and has thus always been prone to regressions.

(See T4175 why this changed in 2.2.12.)

If the default keyserver is not hkps.pool.sks-keyservers.net, then @kristianf's CA certificate has no business certifying it.

I see no need for this.

To be clear, this would allow the least competent CA in the system root trust anchor list to certify an arbitrary server as a member of hkps.pool.sks-keyservers.net. So it is in some sense a security vulnerability -- it allows for a bypass of the correct authority.

I've just pushed 1c9cc97e9d47d73763810dcb4a36b6cdf31a2254 to the branch dkg-fix-T4593

I recognize that adding network activity to the test suite can be complicated (not all test suites are run with functional network access), but if it is possible to have a unit test or something (that doesn't do network access, but just looks at what the dirmngr *would* have tried somehow?), that would be great. Thanks for looking into this!

Confirmed; that looks like a regression.

A possible exception here is that .onion TLDs should stick with HKP by default

Any word on this? i've pushed a fix for this into debian experimental as a part of 2.2.16-2, but i am concerned that there's no adoption from upstream. If there's a reason that this is the wrong fix, please do let me know!

If we only need it for backward compatibility, then the configuration in gpg.conf should *not* be overriding the preferred, forward-looking form of the configuration (in dirmngr.conf). If it is low priority to fix this, then there will be a generation of GnuPG users and toolchains which deliberately configure the value in gpg.conf instead of dirmngr.conf because they'll know that's the more robust way to do it.