Thanks for testing. It's actually an error of generating _unicode_mapping.c, which utf8.c includes.

Thanks for reporting; the code is really new and not yet fully tested.

It seems that the reference to PKCS#5 is correct. It is an issue of how to describe the case of more than 8-byte padding in OpenPGP.

Your example data is malformed, I suppose.

I also don't think that key size obfuscation is useful, after all the preferences of the key demand a certain key size.

Thanks for following up!

No, we always stated that the user id is a mandatory part of OpenPGP keyblocks and that non-compliant keyblocks are rejected. The only exception we made are for revocation signatures where we allow a standalone packet. That exception is done to allow typing in a printed out revocation signature.

To be clear: marking this ticket wontfix means (among other things) that it is the GnuPG project's upstream position that:

With OpenPGP we made user ids mandatory to avoid problems we had with PGP2. I see no reason to revert this.

Nine months have passed since the patches for this problem have been available.

Okay, in 2.2 the output now looks like this:

Won't happen for 2.2

Given that we may move to yet another format in 2.3 I now doubt that we should add such a feature to 2.2.

For reference, here's an error message from openssl smime when it is trying to verify an e-mail message with no embedded certificate at all (despite it knowing about the relevant certificate):

There are likely some bugs in the new code and I also want to do some improvements; see rGb4f1159a5bd7. But things should basically work as before and thus I set this again to testing

i'd be unlikely to ship anything as /etc/gnupg/gpg.conf or /etc/gnupg/dirmngr.conf just because of the mess that admins have to deal with when shipped config files change.

Arggh, gpgconf uses its own option parser so adding the global config file there will require some extra work.

@dkg You might find this interesting. Debian could do stuff in /etc/gnupg/gpg.conf or /etc/gnupg/dirmngr.conf without patching GnuPG to change some defaults.

All done in master with the latest libgpg-error (see T4859). There is always a global configure file in /etc/gnupg (or whatever "gpgconf --list-dirs sysconfdir" prints). The name of the configure file is the same as the user config file (gpg.conf, gpgsm.conf, gpg-agent.conf, ...) but for gpg.conf no versioned config names are used.

Okay, we now have global conf files in master. The extra flags to ignore or force certain options will be added to libgpg-error.

It has now been over 6 months since the patches were available to fix this problem and they have not been adopted upstream.

That sounds like you might have a different issue in mind?

Figuring out the matching user id for a new key signature. Right, --import-options repair-key is the the default and does the same. However, it was also the major cause for the recent trouble with the keyservers because it tried to verify all signatures. repair-keys was made the default (T2236) because it seemed to be nearly for free - which was a false assumption. We should not use this option by default and only consider properly placed signathures as valid. This of course also means that a userid is required.

@werner, i don't understand your last remark. what "required computations" do you think the proposed patches are "moving" from the server to the client?

I see no reason to move required computations from the server to the client.

@werner Could you please give an update on this? Is there any blocker? Is something missing, which prevents merging (and releasing) this?

In 2.2.18, this fix is not included. (partial fix was reverted)

That is actually a GnuPG thing. We originally did it this way to help people remember their passphrase before they start using the key. I agree it is annoying and I would like to remove it too. At the same time we should really think about making no-passphrase the default and require it only with certain compliance settings.

it's been almost a quarter year since my last nudge on this supplied patch. It's not clear to me why it hasn't been merged in master. I'm trying to not be a nag, but:

I thought I close this after the release of 2.2.18.
Anway, it's done, so, closing.

Is this resolved?

On July, 19th, @werner wrote:

You need to wait a bit more.

Or... it could be a feature, not bug, so that failure of -e -r someone can be examined by --locate-keys someone.

Let me clarify the point.

I think that we should apply further change:

diff --git a/g10/getkey.c b/g10/getkey.c
index 077209415..1c337149c 100644
--- a/g10/getkey.c
+++ b/g10/getkey.c
@@ -1369,7 +1369,7 @@ get_best_pubkey_byname (ctrl_t ctrl, enum get_pubkey_modes mode,
     *retctx = NULL;

I found more wrong cases of get_best_pubkey_byname.
For ranking results,
(1) It may return non-encryption primary key as the most relevant key, when its validity is higher.
(2) It may not select encryption primary key even if its creation time is newer.

I also think this makes the most sense.

In my opinion, --locate-key should locate encryption key.

There are some problems with the definition of --locate-key. Further discussion required.

@werner Yes, that sounds great, and would help already a lot, but extending it for card keys would be optimal. Thanks for your work.

In master (to be 2.3) you can add a Label: line into the sub key file of on-disk keys. I use this for quite some time now to show me alabel for my on-disk ssh keys so that I known which one was requested. We can and should extend this to card keys.

I believe that constraint of ret_keyblock != NULL is OK.
Pushing the fix.
Perhaps, backport to 2.2 should be done, too.

If we can assume ret_keyblock != NULL (it is, in current implementation), it can be as simple as:

diff --git a/g10/getkey.c b/g10/getkey.c
index 6802026f6..27bbd354c 100644
--- a/g10/getkey.c
+++ b/g10/getkey.c
@@ -1354,6 +1354,8 @@ get_best_pubkey_byname (ctrl_t ctrl, enum get_pubkey_modes mode,
   int is_mbox = is_valid_mailbox (name);
   int wkd_tried = 0;

Agreed.

It has now been more than a month since:

Thanks.

i've just pushed rGc4b9eba1d6a63b73238dcbb644b365dc53563f3d to the dkg-fix-T4682 branch resolve this.

Sounds interesting @stm! Are there technical documents or specifications I could read to dig into details?

@dkg First step toward the canonical OpenPGP certificate export: http://git.savannah.nongnu.org/cgit/libtmcg.git/commit/?id=75372cac01501ae427dec1ae18805449bf28d087

@wiktor-k Thanks for your interest.

@werner wrote:

Other tasks in master are right now more important.

Other tasks in master are right now more important. You need to wait a bit more.

So, what about this? If I recall correctly, we had agreed in the call to merge this patch, at least into master?

@gniibe, thank you for backporting this to STABLE-BRANCH-2-2!

@dkg You are right. The term "issuer" was too ambiguous.
I like your proposal and would try to implement it. However, "export" of dkg-keycheck and other programs from DKGPG are very limited.

@stm it kind of is a last-resort already, given that it's only in the event where the signature creation dates are equal, but sure, i wouldn't mind adjusting the proposal to say that (sigs) means "sort by date, then issuer, then binary content" -- but what do we think "sort by issuer" means?

does the removal of the gpg22 tag mean that it will not be possible to rely on colon-delimited output for the gpg 2.2 series?

I am proposing to backport rG33c17a8008c3ba3bb740069f9f97c7467f156b54 and rGa7a043e82555a9da984c6fb01bfec4990d904690 to STABLE-BRANCH-2-2 as they represent a significant performance improvement in several specific use cases and appear to have no downsides.

@gniibe, the documentation (at least on the stable branch) says that --fast-import is just a synonym for --import. is that incorrect?

About importing, there are two other works: repairing and trustdb update. We can figure out the difference by the --import-options of no-repair-keys and fast-import (to skip those works).
I think that both can be O(N^2) for number of signatures.

A linked list of 100000 items is not a usable data structure. The problem however is not the linked list but the DoS due to the number of signatures being well beyond the design limit. 1000 key signatures is already a large number and only few people have them. We need to put a limit on them.

with @gniibe's patches applied, i profiled the --import, since that is where the largest CPU cost remains. I tried two different times:

Okay, for 100000 signature this is clearly a win if no key lookup is needed.

i also checked the CPU time for git tag -v, whether @gniibe's patches were applied or not.

fwiw, i tried gpg --import on the ascii-armored version of my C4BC2DDB38CCE96485EBE9C2F20691179038E5C6 OpenPGP certificate (22895014 octets, 54614 certifications), followed by gpg --list-keys and gpg --export | wc. I was comparing 2.2.17-1 (from the debian package in unstable) with the exact same source, just with @gniibe's two patches rG33c17a8008c3 and rGa7a043e82555 applied as well. I did this with GNUPGHOME set to an otherwise empty directory, where i had done touch pubring.gpg to avoid the keybox format. (the two runs did not share a GNUPGHOME).

For the particular problem of --list-key with pubring.gpg, I think we can say it's fixed.

@werner : Yes, the way to go is having something like a server for keys; It can remove all unnecessary search/lookup all together.

(i think that rG33c17a8008c3ba3bb740069f9f97c7467f156b54 is also relevant, though it was not tagged with this ticket)

@gniibe -- thank you very much for tracking down these O(N^2) operations and cleaning them up. I will profile the effect of those changes and report my findings.

We as GPGTools would also like to see this addition being integrated into GnuPG, since we do plan to switch to keys.openpgp.org in the near future, as we have long been hoping for a key server with better performance and among other things email verification. Without this change, revocations would not work as expected in combination with hagrid however. Preferably of course in the 2.2.X branch.

@gniibe: I doubt that your fix really makes a difference. The majority of time is spend on searching the keyring for keys. This is why I have the gpgk thing in the works.

then they are sorted by their binary content.

and from my understanding they are sending the self-signatures anyway.