gpg 2.2 fails to download keys from a keyserver
Closed, WontfixPublic
Actions

Assigned To

Authored By

	jaymzh
	Jul 3 2019, 7:08 AM

Description

EDIT: Originally I thought it was just on new keyrings, so this bug was originally around creating keyrings, but I was mistaken.

This:

gpg --no-default-keyring --keyring /tmp/test.gpg --recv-key $keyid

Used to create a new keybox (/tmp/test.gpg) and import $keyid from the keyserver. Now it creates an empty keybox and does nothing with it:

$ gpg --no-default-keyring --keyring /tmp/test.gpg --recv-key $keyid
gpg: keybox '/tmp/test.gpg' created
gpg: key XXXXXXXXXXXXXXXX: no user ID
gpg: Total number processed: 1
$ gpg --no-default-keyring --keyring /tmp/test.gpg --fingerprint
$

Importing from a local file does, however still work.

It turns out that even if a keyring *exists* and has keys on it, it still doesn't do anything:

$ gpg --no-default-keyring --keyring /tmp/test.gpg --fingerprint DA94104883842826
gpg: error reading key: No public key
$ gpg --no-default-keyring --keyring /tmp/test.gpg --recv-key DA94104883842826
gpg: key DA94104883842826: no user ID
gpg: Total number processed: 1
$ gpg --no-default-keyring --keyring /tmp/test.gpg --fingerprint DA94104883842826
gpg: error reading key: No public key

This is problematic when trying to create, say, a new keyring for a keysigning party.

Details

Version: 2.2.13

Related Objects

Mentioned In: T4393: GnuPG should always accept key updates even if the update does not contain UIDs
Mentioned Here: T4393: GnuPG should always accept key updates even if the update does not contain UIDs

Event Timeline

jaymzh created this task.Jul 3 2019, 7:08 AM

jaymzh renamed this task from gpg 2.2 fails to create a new keybox properly to gpg 2.2 fails to download keys from a keyserver.Jul 3 2019, 7:11 AM

jaymzh updated the task description. (Show Details)

I do not understand your problem: The keyserver does not carry or is willing to send you the requested key. Note that keyservers are for a year now under heady DoS attack and only a few are remaining. I will close this report, please re-open if you figure that it might be a bug in GnuPG.

If you want to look into the details of the keyserver communication put

log-file /tmp/foo/somefile
verbose
debug ipc,network

into ~/.gnupg/dirmngr.conf and restart dirngr.

@jaymzh, I agree with @werner that without more information, this bug report isn't particularly actionable.

I also note that the error message you're seeing ("no user ID") suggests that you may be interested in T4393, which is about merging userid-free certificates with existing certificates.

i also note that if a keyserver is sending you a certificate with no User ID, and you've got an empty keyring, it's not necessarily wrong that you end up with an empty keyring: rfc 4880 explicitly states that an OpenPGP certificate ("transferable public key") expects at least one user ID.

I know the keyservers have been under attack, I'm using 'keys.openpgp.org' which is supposed to be more resilient to these, as I understand it?

I uploaded two keys to it... 58E11BB1E414D9AD and DA94104883842826 using gpg --send-key --keyserver keys.openpgp.org $id

Why keys.openpgp.org doesn't have a keyid for those... is beyond me.

I've included the dirmngr log below... it just looks like HTTP to me... I don't see anything obvious saying a key was found or not found.

However, now that I try it with an SKS keyserver, it all works as expected:

$ rm /tmp/test.gpg
$ gpg --no-default-keyring --keyring /tmp/test.gpg --keyserver pool.sks-keyservers.net --recv-key $keyid
gpg: keybox '/tmp/test.gpg' created
gpg: key 58E11BB1E414D9AD: 520 signatures not checked due to missing keys
gpg: key 58E11BB1E414D9AD: public key "..." imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1
$ gpg --no-default-keyring --keyring /tmp/test.gpg --fingerprint /tmp/test.gpg
-------------
pub   rsa4096/58E11BB1E414D9AD 2013-09-10 [SC]
      Key fingerprint = 121B DA2D 4ACB 6361 6B36  7A0E 58E1 1BB1 E414 D9AD
uid                 [ unknown] ...
uid                 [ unknown] ...
sub   rsa4096/977CDA6871E6B6A4 2013-09-10 [E]
      Key fingerprint = 787F F436 11C2 A41E 0CE7  3D21 977C DA68 71E6 B6A4
$

(I removed the UID info there to prevent email harvesting only).

So... perhaps this is just an issue with the new keyservers? Here's where it feels like there's still a gnupg bug... looking at the dirmngr logs, they both seem to send a key, they both appear to send a full key:

2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Content-Disposition: attachment; filename="58E11BB1E414D9AD.asc"'

and

2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Content-disposition: attachment; filename=gpgkey.asc'

Here's the full logs, first openpgp:

2019-07-03 13:35:07 dirmngr[3901.6] handler for fd 6 started
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 -> # Home: /home/phil/.gnupg
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 -> # Config: /home/phil/.gnupg/dirmng
r.conf
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 -> OK Dirmngr 2.2.13 at your service
2019-07-03 13:35:07 dirmngr[3901.6] connection from process 3900 (1000:1000)
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 <- GETINFO version
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 -> D 2.2.13
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 -> OK
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 <- KEYSERVER --clear hkps://keys.open
pgp.org
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 -> OK
2019-07-03 13:35:07 dirmngr[3901.6] DBG: chan_6 <- KS_GET -- 0x58E11BB1E414D9AD
2019-07-03 13:35:07 dirmngr[3901.6] resolve_dns_addr for 'keys.openpgp.org': 'keys.op
enpgp.org' [already known]
2019-07-03 13:35:07 dirmngr[3901.6] resolve_dns_addr for 'keys.openpgp.org': 'keys.op
enpgp.org' [already known]
2019-07-03 13:35:07 dirmngr[3901.6] number of system provided CAs: 128
2019-07-03 13:35:07 dirmngr[3901.6] DBG: Using TLS library: GNUTLS 3.6.7
2019-07-03 13:35:07 dirmngr[3901.6] DBG: http.c:connect_server: trying name='keys.ope
npgp.org' port=443
2019-07-03 13:35:07 dirmngr[3901.6] DBG: http.c:1899:socket_new: object 0x00007f7dfc2
d6610 for fd 7 created
2019-07-03 13:35:08 dirmngr[3901.6] DBG: http.c:request:
2019-07-03 13:35:08 dirmngr[3901.6] DBG: >> GET /pks/lookup?op=get&options=mr&search=
0x58E11BB1E414D9AD HTTP/1.0\r\n
2019-07-03 13:35:08 dirmngr[3901.6] DBG: >> Host: keys.openpgp.org\r\n
2019-07-03 13:35:08 dirmngr[3901.6] DBG: http.c:request-header:
2019-07-03 13:35:08 dirmngr[3901.6] DBG: >> \r\n
2019-07-03 13:35:08 dirmngr[3901.6] DBG: http.c:response:
2019-07-03 13:35:08 dirmngr[3901.6] DBG: >> HTTP/1.1 200 OK\r\n
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Server: nginx/1.14.2'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Date: Wed, 03 Jul 2019 20:35:08 GMT
'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Content-Type: application/pgp-keys'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Content-Length: 3021'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Last-Modified: Mon, 01 Jul 2019 20:
54:04 GMT'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Connection: close'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'ETag: "5d1a72ec-bcd"'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Content-Disposition: attachment; fi
lename="58E11BB1E414D9AD.asc"'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Access-Control-Allow-Origin: *'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: 'Accept-Ranges: bytes'
2019-07-03 13:35:08 dirmngr[3901.6] http.c:RESP: ''
2019-07-03 13:35:08 dirmngr[3901.6] DBG: chan_6 -> S SOURCE https://keys.openpgp.org:
443
2019-07-03 13:35:08 dirmngr[3901.6] DBG: (3021 bytes sent via D lines not shown)
2019-07-03 13:35:08 dirmngr[3901.6] DBG: chan_6 -> OK
2019-07-03 13:35:08 dirmngr[3901.6] DBG: chan_6 <- BYE
2019-07-03 13:35:08 dirmngr[3901.6] DBG: chan_6 -> OK closing connection
2019-07-03 13:35:08 dirmngr[3901.6] handler for fd 6 terminated

vs SKS:

2019-07-03 13:41:58 dirmngr[3901.6] handler for fd 6 started
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> # Home: /home/phil/.gnupg
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> # Config: /home/phil/.gnupg/dirmngr.conf
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> OK Dirmngr 2.2.13 at your service
2019-07-03 13:41:58 dirmngr[3901.6] connection from process 4086 (1000:1000)
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 <- GETINFO version
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> D 2.2.13
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> OK
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 <- KEYSERVER --clear hkps://hkps.pool.sks-keyservers.net:443
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> OK
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 <- KS_GET -- 0xB191D4580FB203F2
2019-07-03 13:41:58 dirmngr[3901.6] DBG: Using TLS library: GNUTLS 3.6.7
2019-07-03 13:41:58 dirmngr[3901.6] DBG: http.c:connect_server: trying name='2001:41d0:800:d1e::82:0' port=443
2019-07-03 13:41:58 dirmngr[3901.6] DBG: http.c:1899:socket_new: object 0x00007f7dfc00bb90 for fd 8 created
2019-07-03 13:41:58 dirmngr[3901.6] DBG: http.c:request:
2019-07-03 13:41:58 dirmngr[3901.6] DBG: >> GET /pks/lookup?op=get&options=mr&search=0xB191D4580FB203F2 HTTP/1.0\r\n
2019-07-03 13:41:58 dirmngr[3901.6] DBG: >> Host: hkps.pool.sks-keyservers.net\r\n
2019-07-03 13:41:58 dirmngr[3901.6] DBG: http.c:request-header:
2019-07-03 13:41:58 dirmngr[3901.6] DBG: >> \r\n
2019-07-03 13:41:58 dirmngr[3901.6] DBG: chan_6 -> S PROGRESS tick ? 0 0
2019-07-03 13:41:59 dirmngr[3901.6] DBG: http.c:response:
2019-07-03 13:41:59 dirmngr[3901.6] DBG: >> HTTP/1.1 200 OK\r\n
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Date: Wed, 03 Jul 2019 20:41:59 GMT'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Content-Type: application/pgp-keys; charset=UTF-8'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Content-Length: 2194'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Connection: close'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Server: sks_www/1.1.6'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'X-HKP-Results-Count: 1'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Content-disposition: attachment; filename=gpgkey.asc'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Access-Control-Allow-Origin: *'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'X-Cache-Status: MISS'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: 'Via: 1.1 pgpkeys.eu (nginx)'
2019-07-03 13:41:59 dirmngr[3901.6] http.c:RESP: ''
2019-07-03 13:41:59 dirmngr[3901.6] DBG: chan_6 -> S SOURCE https://[2001:41d0:800:d1e::82:0]:443
2019-07-03 13:41:59 dirmngr[3901.6] DBG: (2194 bytes sent via D lines not shown)
2019-07-03 13:41:59 dirmngr[3901.6] DBG: chan_6 -> OK
2019-07-03 13:41:59 dirmngr[3901.6] DBG: chan_6 <- BYE
2019-07-03 13:41:59 dirmngr[3901.6] DBG: chan_6 -> OK closing connection
2019-07-03 13:41:59 dirmngr[3901.6] handler for fd 6 terminated

What am I missing?

I think what you're missing is the keys.openpgp.org documentation which makes it clear that they will not distribute identity information (read: "User IDs") without an explicit confirmation by the operator of the e-mail address named in the User ID. They strip down the certificate pretty significantly before redistribution, especially if the e-mail address hasn't been confirmed directly with the operators of that server.

So you can upload whatever you like, but keys.openpgp.org will spit back out only a stripped-down, minimal certificate.

This is not an issue with gpg itself, it is a design goal of that keyserver implementation.

Aha, thank you. Sorry I saw the original post about the flood attacks (https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f) which said to change your keyserver and I did, but I hadn't realized there were such significant differences.

OK, so I spent some time reading those docs, and trying this a few different ways, validating my UID info, etc. So I don't know there's a solid bug here, but there are a few things that I think could be better from gnupg's side:

The messaging here implies the key was imported:

gpg: key DA94104883842826: no user ID
gpg: Total number processed: 1

But it wasn't. GPG should, IMO, be clear that it skipped this key.

Programmatically, within PIUS (https://github.com/jaymzh/pius) I can parse IMPORT_RES, or at least see the lack of an IMPORT_OD... however, I'd like to be able to tell the user why the import failed, especially in the case that a stripped key was sent back, being able to say "this key doesn't have UID information on it" is useful - especially when automating this stuff, pius-keyring-mgr can just email the relevant user and tell them (like we tell them now if the key wasn't found at all).

Ideally there would be a way to import keys even without UIDs.

Re 1.: I don't view this as a bug. gpg prints stats on what it has been done and clearly it has processed a key. If it would have imported the key you would see another stat line telling about this. There was however a bug in the stats output which has been fixed.

Re 2.: Returning all information why something failed would require a lot of extra code and future maintenance. Thus I doubt that we will add more than diagnosic messages. Note that GPGME recently received a feature to collect diagnostics from gpg and return them to the caller. Kleopatra uses this to show detailed error information if the user requests this.

Re 3.: I am still unconvinced about the use cases for accepting keys without user-id. In contrast to pgp-2, gpg has always had the policy to allow only keys with a user-id and futher only with valid, ie. signed, user-ids. The reason for this is that properties of the key are carried in the self-signature of the user-id. It is possible to take them also from so-called direct-key-signatures, but we do not have much experience with that feature and thus I fear that this will have severe regressions to the ecosystem. RFC-4880 even requires the user id and the planned change in rfc4880bis is merely for a very limited use case on embedded devices where there is no space for a user-id.

Not every incoming certificate that has no user ID will lack a user ID once it is merged with the local copy of the same certificate. T4393 describes that use case, so if you're interested in receiving user-ID-lacking updates to certificates that you already have a copy of, @jaymzh, you should follow up on that ticket.

jaymzh mentioned this in T4393: GnuPG should always accept key updates even if the update does not contain UIDs.Jul 4 2019, 10:33 PM

gpg 2.2 fails to download keys from a keyserverClosed, WontfixPublicActions

Description

Details

Related Objects

Event Timeline

gpg 2.2 fails to download keys from a keyserver
Closed, WontfixPublic
Actions