dirmngr fails with HTTP 302 redirection to hkps
Open, HighPublic

Description

When dirmngr receives an HTTP 302 redirection from an hkps server, it closes the first connection and then appears to re-try a subsequent connection in a new TCP stream, but it fails because the packet is a perhaps is trying to re-use the same TLS state.

my dirmngr.conf in this test says:

debug-level guru
debug-all
log-file /tmp/cdtemp.BOwDTQ/dirmngr.log
keyserver hkps://dkg.fifthhorseman.net
no-use-tor

I've attached the dirmngr.log generated in this session, and the packet capture of the session too, which shows the weird encrypted data handshake packet.


Details

Version
2.2.16
dkg created this task.Sat, Jun 8, 6:53 AM
werner triaged this task as High priority.
werner added a project: gnupg (gpg22).
dkg added a subscriber: werner.Sat, Jun 8, 2:20 PM

thanks for the triage, @werner!

I notice that you've tagged this ntbtls -- but the version of dirmngr i was using is built with gnutls.

I hope we'll be able to figure out a resolution for it using gnutls as well (that is, that doesn't involve requiring a switch to ntbtls), because i'd like to backport the fix to the upcoming debian release (probably GnuPG 2.2.12 or 2.2.13, given the date of the debian freeze earlier this year), and we're far enough along in that release cycle that a wholesale switch of TLS engines is likely to be too large of a change to be acceptable to the release team.

dkg added a comment.EditedSat, Jun 8, 4:16 PM

fwiw, the bug looks like it's in send_request in ks-engine-hkp.c, which re-uses the http_session object without re-initializing its tls_session member.

The lowest latency fix to this problem would be to not close the TLS session or the TCP connect, and just pipeline another request on the same channel. But some HKP servers probably can't handle pipelining, and if the host is a different host, that is also probably not possible (you could re-use the channel for a different host if the server's X.509 cert covers both hosts, like www.example.com doing a 302 redirect to example.com, but that's not the general case).

So in at least some cases, dirmngr will need to fully tear down the tls_session object and re-initialize it before trying the redirect.

If I understand correctly, this is exactly the same problem that the one we encountered some time ago in the code dealing with fetching keys from HTTP (--fetch-keys), and that we fixed with this patch.

The relevant logic in ks-engine-hkp.c is very similar to the one in ks-engine-http.c so we could apply the same kind of fix. I will submit a patch to that effect.

I just assumed that is an ntbtls problem.

dkg added a comment.Tue, Jun 11, 9:41 AM

@gouttegd good catch!

It's frustrating that the code duplication between ks-engine-http.c and ks-engine-hkp.c meant that the earlier fix didn't cover this case.

For STABLE-BRANCH-2-2, I've pushed branch dkg-fix-T4566 with d4dc1ee403c5fd09ebcf5de213b940d35b8337f2 and have tested that it does resolve this error. I haven't tried to apply it to master.

dkg added a comment.Wed, Jun 19, 7:06 PM

Any word on this? i've pushed a fix for this into debian experimental as a part of 2.2.16-2, but i am concerned that there's no adoption from upstream. If there's a reason that this is the wrong fix, please do let me know!