wow, 46MiB, that's even worse than mine. :( thanks for sharing the update, @jackalope. I'm glad you've worked around it for now, but sadly this kind of certificate flooding could happen at any time if you're using the SKS keyserver network :(

That's a great question, @jackalope. I found this in a different misbehaving keyring recently by basically deleting keys by hand until only one was left. surprise, it was mine (ugh)! But that process is pretty slow and manual and tedious. Let me see if i can do better.

@jackalope, the place where the output is hanging is likely due to output buffering (i have been able to replicate the same problem, and the output hangs at intervals of 8192 octets). So while it is giving you a clue about where the hang is, it's not a very precise clue.

Thanks for the feedback, @werner. I think I understand the reasons that we've gotten to this place -- but that doesn't mean i think it's ok to stay here. In this bug report, i'm pointing out that the documentation and the feedback/error reporting is misleading, which leads to difficulty in debugging. We need to do something about it.

I note that this is likely happening because we are using gcr's system-modal prompter. I haven't looked into whether it's even possible to use gcr in a non-system-modal way, but i'd welcome pointers.

i think this might be a duplicate of T4496

I'm unlikely to put a windows-specific patch into the debian source, as
i have no good way of testing it, and it wouldn't affect any binary that
we ship.

Hm, T4521 suggests that the two different cases should not be treated differently. If you think that they *should* cause distinct behavior, please do mention it over there!

@gniibe, thanks for the diagnosis! I agree that restarting or shutting down the backends should be done in the reverse order as a simple workaround.

without feedback, i have no idea what you want to do here as upstream. I believe this issue has identified a specific failing use case, and it has a patch that fixes the problem. if there's a problem, please let me know what it is. If there's no problem, please consider merging.

I note that "the best" seems like it might be a pretty subjective thing. The standard GnuPG framing asks about the validity of keys for the User ID in question. Perhaps the caller could indicate whether they want to require full validity for each key to make this key selection more strict.

The function would do something like:

• from msg, extract all e-mail addresses from to, cc, bcc fields
• find "the best" keys that match these addresses, storing them in keylist
• copy msg to tmp, remove bcc header from tmp
• wrap armored output of gpg.Context.encrypt(bytes(tmp), recipients=keylist) in the necessary RFC 3156 cladding, copying most headers from msg (maybe stubbing out the subject), producing an email.message.EmailMessage object.

Any word on this? i've pushed a fix for this into debian experimental as a part of 2.2.16-2, but i am concerned that there's no adoption from upstream. If there's a reason that this is the wrong fix, please do let me know!

we now have a DCO from @Valodim

If we only need it for backward compatibility, then the configuration in gpg.conf should *not* be overriding the preferred, forward-looking form of the configuration (in dirmngr.conf). If it is low priority to fix this, then there will be a generation of GnuPG users and toolchains which deliberately configure the value in gpg.conf instead of dirmngr.conf because they'll know that's the more robust way to do it.

@werner, My usual approach for private branches is to prefix with dkg/, but (a) playfair rejects branch names with a /, and (b) i'm not the author of these patches, and i didn't want to claim credit that doesn't belong to me.

I've pushed @Valodim's proposed patches to the fix-4393 branch in our git repo. they look good to me, and i think they should be merged to master.

I think this commit should be reverted -- if the test fails we should figure out why and fix it, because the logic of the test is correct.

It also passes for me with python 2.7.16 (debian package 2.7.16-2).

i think you mean t-decrypt-verify.py, right? That seems to indicate a problem on the targeted system that we ought to fix, rather than just commenting out the test. t-decrypt-verify.py passes for me when i test it with python 3.7.3 (debian python 3.7.3-1). what version of python are you testing with?

Sorry for the truncated commit. the sentence should have been:

@gouttegd good catch!

fwiw, the bug looks like it's in send_request in ks-engine-hkp.c, which re-uses the http_session object without re-initializing its tls_session member.

thanks for the triage, @werner!

thanks for fixing that error message, @werner. As @Valodim points out in discusson about hagrid, a gpg.conf keyserver option (deprecated according to the documentation) overrides the dirmngr.conf keyserver option (not deprecated according to the documentation.

any feedback on this proposed patch?

fwiw, i'm used to using slashes in my branch names in dozens of other projects. I was trying to keep my branches scoped under dkg/ so that others could ignore them if they wanted. If the only issue is that i need to not do that, i'm fine naming them with hyphens instead of slashes (or whatever). I'll use that rule for future work.

@gniibe thank you!

I've pushed fa0a5ffd4997c2ca38a1dd2d89459b6b1f18ad99 to the branch dkg/fix-T3464, which i think solves the problem i was seeing without reintroducing any new problems.

I can confirm that this is actually a problem now :( gpgme_op_decrypt_verify returns a status with GPG_ERR_MISSING_KEY set when a session-key is used.

Perhaps i wasn't clear enough in the earlier messages on this thread. The inclusion of restrictively-licensed code in a file that also claims LGPL/GPL appears to be an unredistributable license. Could you please clarify why the GPL or LGPL applies to libksba while it contains src/cms.asn in its current form?

we've never shipped a binary gpgscm in any debian package. I was just reviewing the differences between what we ship and what upstream ships, and i noticed this discrepancy.

By marking this as "wontfix", you appear to be saying that you won't even fix the documentation to describe the constraints that gcrypt intends to enforce. This is surprising to me.

And yet, that interface is already being used by the agent-transfer utility in monkeysphere. The interface exists, it is not marked in any way as unusable or deprecated or off-limits, so it is used.

trigger what command? i'm pretty sure gpgconf --reload gpg-agent does not trigger updatestartuptty. And it should not do so, afaict -- if you think it should, i'd be interested in hearing the rationale for it.

This doesn't sound systemd-specific to me, fwiw, though i don't understand how to reproduce the problem from the given description here.

"requires too much changes" i can understand.

(hm, i'm pushing apparently successfully to playfair.gnupg.org:/git/libgcrypt.git but it is not showing up here. if you want to fetch this patch, you can also find it on the http-to-https branch at https://gitlab.com/dkg/libgcrypt.git

I think you are saying that dirmngr receives the query term as escaped data in the assuan connection from the dirmngr client (typically, gpg, which itself decides how to percent-escape what it feeds into libassuan).

I think you'll be better off doing this with the simpler --quick-generate-key and --quick-add-key interfaces, rather than hacking on the domain-specific language used by --batch --generate-key.

I think this patch should be backported to STABLE-BRANCH-2-2

I think this patch should be backported to STABLE-BRANCH-2-2

I can confirm that this fix repairs the problem on debian's s390x.

I've just pushed e4a158faacd67e15e87183fb48e8bd0cc70f90a8 to branch dkg/fix-T4501 as a proposed fix for this specific problem (it doesn't introduce anything in the test suite, or try to deal with any of the other %b problems).

OK, i think the reason this is happening is that agent_public_key_from_file (in agent/findkey.c) is screwing up a %b format string in gcry_sexp_build_array.

Ok, the difference appears to be that on these 64-bit big-endian platforms, they're returning a zero-byte string for the associated comment. When this happens, gcry_sexp_canon_len returns 0 because of GPG_ERR_SEXP_ZERO_PREFIX. The same thing happens on x86_64 platforms when confronted with such an s-expression.

It looks to me like gcry_sexp_canon_len is returning 0 on these platforms from within a backtrace like this:

I've just pushed 29adca88f5f6425f5311c27bb839718a4956ec3a to the dkg/fix-T4490 branch, which i believe fixes this issue.

Validity values are also displayed for all user IDs.
[…]

show-uid-validity
       Display  the  calculated  validity of user IDs during key
       listings.  Defaults to yes.

[…]

Trust values are used to indicate ownertrust and validity of  keys  and
user IDs.  They are displayed with letters or strings:

[…]

revoked
       For validity only: the key or the user ID has been revoked.

@werner, why is it the case that if i'm willing to look up a key via WKD on Monday, i should by definition also be willing to send a followup request to that WKD server on Thursday just because the certificate is marked with an expiration?

And, i just discovered that when i manually edit the key to remove the (comment) list from the *.key S-expression file, the final --export-secret-key works fine. so the failure appears to be due to the presence of the (comment) clause. (same as in T4501)

And, i just discovered that when i manually edit the key to remove the (comment) list from the *.key S-expression file, everything works fine on s390x. so the failure appears to be due to the (comment), just like in T4490.

fwiw, i've just tried loading the same keyfile that the s390x (64-bit big-endian) implementation choked on into a running gpg-agent on an amd64 machine (64-bit little-endian) and gpg --full-generate-key succeeded with that same key on amd64.

This is particularly bad for users who have manually specified a given keyserver in dirmngr.conf, because even a transient failure in that keyserver will prevent them from any future keyserver requests until dirmngr decides that the "death" has worn off.

further testing suggests that the invalid URI issue is only present for dirmngr's --keyserver option, and gpg's deprecated --keyserver option actually accepts schema-less hostnames.