I know this problem very well and it let to the introduction the import filters. For example I can update my own key only using filters like

import-filter drop-sig=   sig_created_d=2015-12-24
import-filter drop-sig=|| sig_created_d=2016-03-16

in my gpg.conf. The OpenPGP format as used in the past is subject to easy DoS regarding key signatures. Some people had the very same problem already with PGP-2. The only useful mitigation I can see is to allow the im,port of key signatures only from trusted source and not from keyservers.

i'm aware of the filters you're using, but they are not a principled response to this kind of certificate flooding attack. An attacker who wants to be really abusive can easily create certifications that bypass any import-filter gpg is capable of.

In the body of the message above, I describe a specific set of mitigations that would be concretely helpful (filtering incoming keys that are too large before attempting to put them in the keyring). Can you comment on those please? If you think they are not useful, i'm happy to hear an explanation.

With those mitigations, keyservers (not "trusted sources") are still useful for revocation and subkey updates, as long as we have fixed T4393.

Note also that some keyservers like keys.openpgp.org will distribute only verified self-sigs (including revocations and subkey updates) without distributing the floodable third-party certifications. We can and should distinguish "updates-only" keyservers from discovery-by-address mecahnisms.

But yes, if we do that, this might mean that the SKS network falls away. GnuPG currently uses dirmngr's keyserver directive for both --search (by uid) and --recv (by fingerprint or keyid). Can we make that distinct somehow? or just deprecate --search entirely, and encourage --locate-key instead?

I have mentioned it several times in the past that I would like to see the search by user id feature be removed from keyservers so that there is less incentive to use them as a perpetual and searchable database for maybe illegitimate data.

gpg would then do a locate-key instead of a --search-key. Given the current state of things we may remove/redefine --search-keys even without changes by the keyservers.

That is probably not what you want but at least it allows to import your key

$ gpg --recv-key -v --keyserver-options import-self-sigs-only 0xC4BC2DDB38CCE96485EBE9C2F20691179038E5C6
gpg: data source: https://51.38.91.189:443
gpg: armor header: Version: SKS 1.1.6
gpg: armor header: Comment: Hostname: pgpkeys.eu
gpg: key F20691179038E5C6: number of dropped non-self-signatures: 54614
gpg: pub  ed25519/F20691179038E5C6 2019-01-19  Daniel Kahn [...]
gpg: key F20691179038E5C6: 4 duplicate signatures removed
gpg: key F20691179038E5C6: 4 signatures reordered

I will see what else can be done as a quick solution. Regarding my former comment: At that time I had not yet read the thread about this problem.

thanks for working on this @werner. rG2e349bb61737 is definitely not useful for me. If i am going to tell anyone "hey, do this weird thing differently in order to fetch my key", i will tell them "pull it from https://dkg.fifthhorseman.net/dkg-openpgp.key". I will never tell anyone to use import-self-sigs-only.

Not only that, but the current implementation of import-self-sigs-only also does not appear to be robust against a malicious certificate flood given SKS's lack of cryptographic validation. Adding a new option to an already-crowded space is not the right solution. the right solution is for gpg to be more defensive about the OpenPGP packets it receives, regardless of who it receives them from. Again, i described a series of fallbacks that (a) don't require the user to set any options, and (b) make more sense than import-self-sigs-only in the first place. Could you please comment on that proposal?

I've opened T4599 about deprecating --search explicitly, so we should have that separate discussion over there.

Even if you can't use it the option is still useful to avoid other kinds of DoS. As written in the comments it is not a full solution but it helps to side-step issues with key-signature. In particular for sites which do not have a need for them.
BTW, revocation certificates are still merged with the new option.

The problem you see with your key is the keyblock limit of 5 MiB we use with keyboxes. This has been been set to avoid, surprise, DoS. Now we could increase that limit but it is not really helpful. You suggestions are to try to keep the keyblock within that limit using a several step fallback scheme. This is quite some overhead so I hesitate to implement it in this way.

But we can do a it with one fallback step: If we run into the image-larger-than-5MiB error, we try again but this time using the new option. In contrast to --import-clean this is much faster than running the full import step again and we can do an import-clean anyway in this case. I need to check the details, though.

I implemented that in master. The first output is from an update of your key and the second from an insert of a new key.

gpg: pub  ed25519/F20691179038E5C6 2019-01-19  Daniel Kahn Gill[...]
gpg: key F20691179038E5C6: 4 duplicate signatures removed
gpg: key F20691179038E5C6: 54614 signatures not checked due to missing keys
gpg: key F20691179038E5C6: 4 signatures reordered
gpg: error writing keyring '[...]/pubring.kbx': Provided object is too large
gpg: the keyblock was too large, retrying with only self-sigs
gpg: pub  ed25519/F20691179038E5C6 2019-01-19  Daniel Kahn Gill[...]
gpg: key F20691179038E5C6: number of dropped non-self-signatures: 54614
gpg: key F20691179038E5C6: "[User ID not found]" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

gpg: pub  ed25519/F20691179038E5C6 2019-01-19  Daniel Kahn Gill[...]
gpg: key F20691179038E5C6: 4 duplicate signatures removed
gpg: key F20691179038E5C6: 54614 signatures not checked due to missing keys
gpg: key F20691179038E5C6: 4 signatures reordered
gpg: error writing keyring '[...]/pubring.kbx': Provided object is too large
gpg: the keyblock was too large, retrying with only self-sigs
gpg: pub  ed25519/F20691179038E5C6 2019-01-19  Daniel Kahn Gill[...]
gpg: key F20691179038E5C6: number of dropped non-self-signatures: 54614
gpg: using pgp trust model
gpg: key F20691179038E5C6: public key "[User ID not found]" imported
gpg: Total number processed: 1
gpg:               imported: 1

Note the final stats. It takes some time to import and process the key and all its signatures during the first phase but the fallback is then much quicker. The fallback uses --import-clean,self-sigs-only. I need to cleanup a few string and will then commit to master. More on this the tomorrow.

Thanks for working on this fallback, Werner.

On the issue of flooding attacks:

• if an adversary first provides a certificate that is just barely under 5MiB, it will be imported in full, correct?
• if such a certificate is imported, and then in a separate fetch, gpg receives a certificate with the same primary key, but with a revocation (perhaps even just a two-packet certificate, with just a primary key and a revocation certificate, or even just a raw revocation certificate), what happens? The aggregated certificate (including the revocation) will be > 5MiB.

Does this limit mean that an attacker can block a GnuPG user from receiving and importing a revocation by padding out their certificates to 5MiB?

as a separate variant: if the attacker floods the certificate with bogus self-signatures -- that is, certifications that have an issuer fingerprint or issuer key id subpacket, whether hashed or unhashed -- will that make it impossible to import any of them?

my initial scenario is where an adversarial keystore floods a certificate right up to (but within) the 5MiB boundary, so that the user has stored it in the keyring already. Then, the user encounters the certificate again, with revocation attached.

Are you saying that if the fallback case is hit during import, then all previously-stored third-party certifications will also be stripped from the keyring?

If so, that strikes me as a bad outcome: an adversarial keystore can force the removal of all locally-stored third-party certifications, potentially making the certificate no longer valid.

If not, then it seems that the certificate + revocation will be above the 5MiB boundary, which means it will be rejected before storage, leaving the user with a non-revoked certificate. This is also a bad outcome (albeit in a different direction).

Okay, if an attacker exactly hist that limit your case is valid. I see no easy fix here, though. What we can do is what is done on Unix file systems to give average users a disk full erroreven if there a few percent of the disk is free; root can use that extra space then. Revocation certificates would be what root is on Unix file systems.

Needs a bit of work in gpg; the question is whether it can wait until the after next release.

The fact that import-clean modifies already-held certifications makes me think it is inappropriate to have as the default for keyserver access (see T4628 for more details).

Given that self-sigs-only is enabled by default, what additional value does import-clean have for the user?

Is the attempt to defend against an abusively large list of self-sigs?