My plan is to let --search-key be the same as locate-key but without local lookups, thus it will be the same as

That was pretty easy to reproduce thanks to your still not working server.

I did some manual tests using netcat and KS_FETCH to test the redirection.

I think you're suggesting accepting *any* path if the hostname of the proposed redirection matches openpgpkey.example.org when querying the WKD direct URL for an @example.org address. That would also be a fine solution from my point of view.

I head the same idea when I read your configuration. Given that the advanced lookup was not reallydeployed (see T4590) I also expect that we will receive complains now that it works. Thus white listing any "openpgpkey." seems to me a reasonable easy solution.

Will be in 2.2.17

Oh dear, that happens if one is always on master. I simply forgot to cherry pick the change from master back in November.
Two commits, though.

@werner, thanks for the pointer to the report, that's certainly useful. And i'm happy that organizations like SektionEins are doing GnuPG audits and publishing their results regardless of who paid for them.

See https://sektioneins.de/en/blog/18-11-23-gnupg-wkd.html for details. In short they fear that companies using IP based security for internal services can be attacked via redirect request and in particular becuase that can happen in the background without the user noticing. I am not concerned but we had long lasting discussions also with protonmail about this and the result was that we need to have this protection. We do not know who requested and paid for the audit from SektionEins and they won't tell us.

Thanks for the pointer, @werner. Certainly we want T4590 fixed.

We need to rewrite the Location to avoid a CSRF attack. See fa1b1eaa4241ff3f0634c8bdf8591cbc7c464144

I should add that i don't really care whose fault it is if the software is broken by some downstream. if it harms any users, and we can fix it, we should fix it, especially if the fix is easy.

We're writing free software, which we know that people use and modify downstream. if we know that the software has a particular sharp edge that people who are modifying it are likely to cut themselves on, we have two options:

Come on, if someone changes the software and breaks it, it is their's fault ant not ours. The whole thing on which keyserver and certificate to use as been discussed ad nausea in the past. Given all the problems with the keyservers I do not see a reason to change it right away to a state we had before. Keyserver code is pretty hard to test and has thus always been prone to regressions.

(See T4175 why this changed in 2.2.12.)

If the default keyserver is not hkps.pool.sks-keyservers.net, then @kristianf's CA certificate has no business certifying it.

I see no need for this.

To be clear, this would allow the least competent CA in the system root trust anchor list to certify an arbitrary server as a member of hkps.pool.sks-keyservers.net. So it is in some sense a security vulnerability -- it allows for a bypass of the correct authority.

I've just pushed 1c9cc97e9d47d73763810dcb4a36b6cdf31a2254 to the branch dkg-fix-T4593

I recognize that adding network activity to the test suite can be complicated (not all test suites are run with functional network access), but if it is possible to have a unit test or something (that doesn't do network access, but just looks at what the dirmngr *would* have tried somehow?), that would be great. Thanks for looking into this!

Confirmed; that looks like a regression.

A possible exception here is that .onion TLDs should stick with HKP by default

Any word on this? i've pushed a fix for this into debian experimental as a part of 2.2.16-2, but i am concerned that there's no adoption from upstream. If there's a reason that this is the wrong fix, please do let me know!

If we only need it for backward compatibility, then the configuration in gpg.conf should *not* be overriding the preferred, forward-looking form of the configuration (in dirmngr.conf). If it is low priority to fix this, then there will be a generation of GnuPG users and toolchains which deliberately configure the value in gpg.conf instead of dirmngr.conf because they'll know that's the more robust way to do it.

@gouttegd good catch!

I just assumed that is an ntbtls problem.

If I understand correctly, this is exactly the same problem that the one we encountered some time ago in the code dealing with fetching keys from HTTP (--fetch-keys), and that we fixed with this patch.

fwiw, the bug looks like it's in send_request in ks-engine-hkp.c, which re-uses the http_session object without re-initializing its tls_session member.

thanks for the triage, @werner!

We need --keyserver in gpg for just one reason: backward compatibility.

thanks for fixing that error message, @werner. As @Valodim points out in discusson about hagrid, a gpg.conf keyserver option (deprecated according to the documentation) overrides the dirmngr.conf keyserver option (not deprecated according to the documentation.

We only supported SHA-1 signed OCSP requests. Fix will go into 2.2.16.

I doubt that we are going to implement this.

Interesting tinge: The main CRL of the dgn.de CA uses a nextUpdate in the year 2034 (15 years in the future) which would force dirmngr to cache the CRL until then. However, the CRL of the intermediate certificate has a nextUpdate only one month in the future. There is currently no entry in that second level CRL, so their idea might be that an updated second level CRL will also trigger a reload of the main CRL. I have not checked how we implement that in Dirmngr but I doubt that such a thing will work for us and that it is in any way standard compliant.

Are you not reading what I am saying to you?? Once again, your explanation is INVALID because that would mean that gnupg would be BROKEN, because it would be a NON-COMPLIANT http client according to the RFC I quoted.

I explained why the keyserver access requires access to the DNS. If that is not possible the keyserver code will not work. If you don't allow DNS to work you either have to use Tor (which we use to also tunnel DNS requests) or get your keys from elsewhere. Also note that the keyserver network is current several broken and under DoS and thus it is unlikely that it can be operated in the future.

I agree with @dkg here.

"requires too much changes" i can understand.

This requires too much changes and does not reflect the reality. It actually makes debugging harder for us.

Thanks

I think you are saying that dirmngr receives the query term as escaped data in the assuan connection from the dirmngr client (typically, gpg, which itself decides how to percent-escape what it feeds into libassuan).

This is easy to explain: dirmngr receives already escaped data and that is what you see in the log. For proper parsing of the URI the escaping needs to be removed and only before sending the request the required escaping is applied. '@', '<', and '>' do not need to be escaped and thus you see them as they are.

I removed this specialized error message. Thanks for reporting.

This is particularly bad for users who have manually specified a given keyserver in dirmngr.conf, because even a transient failure in that keyserver will prevent them from any future keyserver requests until dirmngr decides that the "death" has worn off.

further testing suggests that the invalid URI issue is only present for dirmngr's --keyserver option, and gpg's deprecated --keyserver option actually accepts schema-less hostnames.

see also T4467

I just noticed that dirmngr(8)'s documentation for its --keyserver option says:

Note that even sending a HUP to dirmngr, when it is in this autodetection mode that observed tor at the start, is insufficient to have it re-run the autodetection. You have to explicitly terminate dirmngr to get it to unlearn the autodetected presence of Tor. This is subtly hinted at in dirmngr(8), but no justification is given for it.

One of the things that dirmngr has going for it is that it tracks the current network state, and it would be nice to be able to reuse that state across sessions. If an ephemeral keyring can't use a shared dirmngr, there are fewer arguments for having dirmngr in the first place, and people might be more justified in replacing it with things like https://gitlab.com/anarcat/scripts/blob/master/openpgp-key-get

I don't anymore think this is a high priority request. BTW, A more real problem than several dirmngr instances is multi-user access to smartcards.

HTTP/1.1 spec, RFC 7230, Section 5.4, paragraph 2:
https://tools.ietf.org/html/rfc7230#section-5.4

Please be so kind and point me to the specs stating that you should put the IP address into Host:

It's up to GPG to send the Host header that shows the user's intent.

So in short you want:

1. Allow to specify a keyserver by IP without any DNS lookups.
2. When connecting via IP use the IP address for Host:.