Quiet tricky to get right; needs some rework.

I closed this as a duplicate of a newer task because I did not find this issue when creating T4561 and there was already work done for T4561.

Not every incoming certificate that has no user ID will lack a user ID once it is merged with the local copy of the same certificate. T4393 describes that use case, so if you're interested in receiving user-ID-lacking updates to certificates that you already have a copy of, @jaymzh, you should follow up on that ticket.

Once a revocation is added (to any part of the certificate), perhaps all the certification packets that are clearly made obsolete by the revocation could be dropped from the certificate? That would certainly free up space to be able to import additional revocations if needed.

Given the recent problems with the keyservers, I expect that the keyserver feature will go away anyway and thus I do not think we will put any more effort into this. Thus I re-tag this as gpg 2.3.

And of course, thanks for your fix.

Applied to both branches. I have run no tests myself, though.

Fix will be in 2.2.17

Fix will be in 2.2.17.
See T4612 for the revocation case.

Re 1.: I don't view this as a bug. gpg prints stats on what it has been done and clearly it has processed a key. If it would have imported the key you would see another stat line telling about this. There was however a bug in the stats output which has been fixed.

Because we use dot-locking in GnuPG and copy-update-write for keyrings. Granted: For gpgv this is not required but the code is identical to the gpg code and adding new code does not make much sense. After all gpgv is a stripped down version of gpg I once wrote for Debian. I see your use case but tehre are other ways to do this and thus anthing here has low priority.

Aha, thank you. Sorry I saw the original post about the flood attacks (https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f) which said to change your keyserver and I did, but I hadn't realized there were such significant differences.

I think what you're missing is the keys.openpgp.org documentation which makes it clear that they will not distribute identity information (read: "User IDs") without an explicit confirmation by the operator of the e-mail address named in the User ID. They strip down the certificate pretty significantly before redistribution, especially if the e-mail address hasn't been confirmed directly with the operators of that server.

I know the keyservers have been under attack, I'm using 'keys.openpgp.org' which is supposed to be more resilient to these, as I understand it?

out of curiosity, why does gpgv need the name of the file?

In that case, you can treat this ticket as a bug in the documentation, which still needs to be resolved.

We need random access and the name of the file. Thus a file descriptor is not sufficient.

Okay, if an attacker exactly hist that limit your case is valid. I see no easy fix here, though. What we can do is what is done on Unix file systems to give average users a disk full erroreven if there a few percent of the disk is free; root can use that extra space then. Revocation certificates would be what root is on Unix file systems.

That was pretty easy to reproduce thanks to your still not working server.

I did some manual tests using netcat and KS_FETCH to test the redirection.

I think you're suggesting accepting *any* path if the hostname of the proposed redirection matches openpgpkey.example.org when querying the WKD direct URL for an @example.org address. That would also be a fine solution from my point of view.

I head the same idea when I read your configuration. Given that the advanced lookup was not reallydeployed (see T4590) I also expect that we will receive complains now that it works. Thus white listing any "openpgpkey." seems to me a reasonable easy solution.

Will be in 2.2.17

Oh dear, that happens if one is always on master. I simply forgot to cherry pick the change from master back in November.
Two commits, though.

my initial scenario is where an adversarial keystore floods a certificate right up to (but within) the 5MiB boundary, so that the user has stored it in the keyring already. Then, the user encounters the certificate again, with revocation attached.

@werner, thanks for the pointer to the report, that's certainly useful. And i'm happy that organizations like SektionEins are doing GnuPG audits and publishing their results regardless of who paid for them.

@jaymzh, I agree with @werner that without more information, this bug report isn't particularly actionable.

I don't think so. The fallback mechnanism will still work and remove everything but valid self-signatures. This gives enough space to write the keyblock with the new revocation certificates. I am not sure about designated revokers in this case.

See https://sektioneins.de/en/blog/18-11-23-gnupg-wkd.html for details. In short they fear that companies using IP based security for internal services can be attacked via redirect request and in particular becuase that can happen in the background without the user noticing. I am not concerned but we had long lasting discussions also with protonmail about this and the result was that we need to have this protection. We do not know who requested and paid for the audit from SektionEins and they won't tell us.

I do not understand your problem: The keyserver does not carry or is willing to send you the requested key. Note that keyservers are for a year now under heady DoS attack and only a few are remaining. I will close this report, please re-open if you figure that it might be a bug in GnuPG.

as a separate variant: if the attacker floods the certificate with bogus self-signatures -- that is, certifications that have an issuer fingerprint or issuer key id subpacket, whether hashed or unhashed -- will that make it impossible to import any of them?

Thanks for working on this fallback, Werner.

Thanks, this is excellent news! I´ll check it, if the new Mailvelope version is available and I´ll let you know about the outcome. If the new version is released, let me know!

Thanks for the pointer, @werner. Certainly we want T4590 fixed.

This seems to be the same issue as the one opened in mailvelope. https://github.com/mailvelope/mailvelope/issues/679.

We need to rewrite the Location to avoid a CSRF attack. See fa1b1eaa4241ff3f0634c8bdf8591cbc7c464144

I cannot do that because all listed above packages are my own products.
Fedora is not execution test suites in more than 90% of all packages so they are not aware of most of the issues exposed by test suites.
Please focus on possible causes of above tests.
I'm opened on any suggestions to make additional diagnostics.

Thanks. You may want to ask on the mailing list gnupg-users to see whether someone else has had problems building on rawhide. Right now we do not have the time for individual support and thus I unfortunately need to prioritize this bug report down.

[tkloczko@barrel SPECS]$ uname -a
Linux barrel 5.1.5-300.fc30.x86_64 #1 SMP Sat May 25 18:00:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[tkloczko@barrel SPECS]$ rpm -q libassuan-devel libcurl-devel libgcrypt-devel libgpg-error-devel libksba-devel libusb-devel npth-devel openldap-devel pcsc-lite-libs gnutls-devel sqlite-devel
libassuan-devel-2.5.3-2.1.fc31.x86_64
libcurl-devel-7.65.1-2.fc31.x86_64
libgcrypt-devel-1.8.4-4.1.fc31.x86_64
libgpg-error-devel-1.36-2.fc31.x86_64
libksba-devel-1.3.5-10.1.fc31.x86_64
libusb-devel-0.1.5-14.fc30.x86_64
npth-devel-1.6-3.fc31.x86_64
openldap-devel-2.4.47-2.2.fc31.x86_64
pcsc-lite-libs-1.8.25-2.1.fc31.x86_64
gnutls-devel-3.6.8-2.fc31.x86_64
sqlite-devel-3.28.0-2.fc31.x86_64

Still about half of the packages are from Fedora rawhide but rest are mine.
Just checked and the test suite fails exactly the same way even started without palatalisation.

Also pushed to 2.2. Right now I can't see what else can be done, so I change the status to testing.

Please share with us the OS used, the versions of the libtaries used and other configuration information.
Also please run again using "make check" without any extra options.

Kleopatra was already using this keyserver at the time i took the screenshots, which is why i opened the ticket since it was working in the command line.

I implemented that in master. The first output is from an update of your key and the second from an insert of a new key.

That won't be easy to debug unless we have intermediate debug values from the generating implementation. That IBM Encryption Facility looks partly similar in the command line options to gpg so I wonder whether it will be possible to get some debug output. @mrdave19: we can continue by private mail in case that is helpful for you (wk at g10code com)

I should add that i don't really care whose fault it is if the software is broken by some downstream. if it harms any users, and we can fix it, we should fix it, especially if the fix is easy.

We're writing free software, which we know that people use and modify downstream. if we know that the software has a particular sharp edge that people who are modifying it are likely to cut themselves on, we have two options:

Come on, if someone changes the software and breaks it, it is their's fault ant not ours. The whole thing on which keyserver and certificate to use as been discussed ad nausea in the past. Given all the problems with the keyservers I do not see a reason to change it right away to a state we had before. Keyserver code is pretty hard to test and has thus always been prone to regressions.

(See T4175 why this changed in 2.2.12.)

Even if you can't use it the option is still useful to avoid other kinds of DoS. As written in the comments it is not a full solution but it helps to side-step issues with key-signature. In particular for sites which do not have a need for them.
BTW, revocation certificates are still merged with the new option.

Hello,

I have no idea – I’m not familiar with cmd.exe – Do you have step by step instructions on how to do this?

Many thanks,

Chris Baillon
Digital Marketing Manager

Mainline Tel: +44 (0)1895 813 000
Fax: +44 (0)1895 200 541
E: chris.baillon@allpondsolutions.co.uk<mailto:chris.baillon@allpondsolutions.co.uk>
[cid:image013.jpg@01D48584.F61733C0]
[cid:image014.jpg@01D48584.F61733C0]https://www.facebook.com/allpond/[cid:image015.jpg@01D48584.F61733C0]https://www.instagram.com/allpondsolutions/

[cid:image016.jpg@01D48584.F61733C0]
[cid:image017.jpg@01D48584.F61733C0]https://www.facebook.com/allpetsolutions/[cid:image018.jpg@01D48584.F61733C0]https://www.instagram.com/allpetsolutions/

W: www.allpondsolutions.co.ukhttp://www.allpondsolutions.co.uk/

W: www.allpetsolutions.co.ukhttp://www.allpetsolutions.co.uk/

All Pet Solutions Ltd is a company registered in England and Wales.
Unit 203| Riverside Way | Uxbridge| UB8 2YF

Please consider the environment before printing this e-mail.

This email, including attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received this email in error please notify the sender and delete it from your system. Emails are not secure and may contain viruses. No liability can be accepted for viruses that might be transferred by this email or any attachment.

All Pet Solutions Ltd. Registered office: 22 Wycombe End, Beaconsfield, Buckinghamshire, England, HP9 1NB. Registered in England and Wales. Registered No. 05678076

From: aheinecke (Andre Heinecke) <noreply@dev.gnupg.org>
Sent: 01 July 2019 07:44
To: Chris Baillon <Chris.Baillon@allpondsolutions.co.uk>
Subject: [Task] [Claimed] T4581: Kleopatra stuck in loading the certificate cache

aheinecke triaged this task as "Normal" priority.
aheinecke claimed this task.
aheinecke added a comment.

Hi,

can you please open the console (cmd.exe) and do a "gpg -K" and "gpgsm -K"

do these commands list all your secret keys or do they hang?

Also please try killing the process "gpg-agent.exe" through the Task manager and try to open Kleopatra again.

Regards,
Andre

TASK DETAIL
https://dev.gnupg.org/T4581

EMAIL PREFERENCES
https://dev.gnupg.org/settings/panel/emailpreferences/

To: aheinecke

Cc: aheinecke, allpond, Neurone, Rafixmod, Fox, ccharabaruk, gp_ast

This is an automated email from the GnuPG development hub. If you have registered in the past at https://bugs.gnupg.org/ your account was migrated automatically. You can visit https://dev.gnupg.org/ to set a new password and update your email preferences.

If the default keyserver is not hkps.pool.sks-keyservers.net, then @kristianf's CA certificate has no business certifying it.

Welcome back from vacation!
@aheinecke Yes I am 1000% sure the passphrase is "dave" without the quotes.
These are the commands I use for the encrypt using the IBM Encryption Facility:

-o '/home/suimgwy/_july1.pbe' \
-s2k-cipher-name AES_256 -s2k-digest-name SHA256 -s2k-mode 3 \
-s2k-passphrase dave \
-t ISO-8859-1 \
-use-mdc \
-c '/home/suimgwy/_input.txt'
<<<

thanks for working on this @werner. rG2e349bb61737 is definitely not useful for me. If i am going to tell anyone "hey, do this weird thing differently in order to fetch my key", i will tell them "pull it from https://dkg.fifthhorseman.net/dkg-openpgp.key". I will never tell anyone to use import-self-sigs-only.

Ping?

That is probably not what you want but at least it allows to import your key

back from vacation so apologies for the delay. @werner This is reproducible on the command line without Kleopatra. So maybe something for you our Gniibe to look into?

I have mentioned it several times in the past that I would like to see the search by user id feature be removed from keyservers so that there is less incentive to use them as a perpetual and searchable database for maybe illegitimate data.

I see no need for this.

Keyserver issues are always hard to analyze because the state of the servers and which server you get is always a factor.

Sorry for the delay, I was on my summer vacation ;-)

can you please open the console (cmd.exe) and do a "gpg -K" and "gpgsm -K"

@joaociocca Can you please try to update to Gpg4win-3.1.9 and try again.

I've just pushed 1c9cc97e9d47d73763810dcb4a36b6cdf31a2254 to the branch dkg-fix-T4593

Note also that some keyservers like keys.openpgp.org will distribute only verified self-sigs (including revocations and subkey updates) without distributing the floodable third-party certifications. We can and should distinguish "updates-only" keyservers from discovery-by-address mecahnisms.

Just importing a ~666KiB certificate when this monster certificate is in the keyring consumes over 10m of CPU time:

Verifying a git tag from the "clean" version of this certificate takes ~225ms of CPU time. Verifying the same git tag from a keyring that contains the flooded version of the certificate takes ~145s. This is factor of more than 600×. Any automated git tag verification system can probably be DoSed by this behavior.

I didn't mean to suggest that switching to sqlite was the only way to fix this, but if it is a promising way to fix it, that would be great. I'm sure there are other ways.

I recognize that adding network activity to the test suite can be complicated (not all test suites are run with functional network access), but if it is possible to have a unit test or something (that doesn't do network access, but just looks at what the dirmngr *would* have tried somehow?), that would be great. Thanks for looking into this!

i'm aware of the filters you're using, but they are not a principled response to this kind of certificate flooding attack. An attacker who wants to be really abusive can easily create certifications that bypass any import-filter gpg is capable of.