All changes in gniibe/t7855 are pushed into master.

Implemented but not tested at all.

Lastly, pushed a change into gniibe/t7855 branch.
rGf861b2a33f96: gpg,gpgsm: Fix thinko for FP closing under no lock.

I pushed further changes into gniibe/t7855 branch.
rG2fe62809014e: gpg,gpgsm: Serialize write access to keybox/keyring to protect.

For remaining changes in 2.2, I pushed changes into gniibe/t7855 branch.
rGbd65b06b74c2: gpg,gpgsm: Don't lock recursively when KEEP_LOCK is enabled.
rG423fd047da87: kbx,gpg,gpgsm: Add FP-close method for keydb to close before unlock.
rG966258ac5f99: gpgsm: Fix delete and store certificate locking glitches.

I pushed changes into gniibe/t7855 for compressing the keybox.
rG8cc2a0e0ffee: gpg: Minor clean up for keydb_lock API.
rGe4d3c3aa2220: kbx,gpg,gpgsm: Introduce keybox_compress_when_no_other_users.
rG3e441d5b299f: kbx,gpg,gpgsm: More changes for compressing the keybox.

Then, we need to integrate following commits of 2.2 into gniibe/t7855 branch:
rG43fe9073aa81: gpg,gpgsm: Tweak the locking of the pubring.kbx
rG8491aca73cff: gpg: Revert the always locking introduced with 43fe9073aa
rGad4a5117ab1c: gpgsm: Properly release the lock when compressing a pubring.
rG7962eca3a023: gpgsm: Change delete and store certificate locking glitches.
rG22f9c4a3b3c1: gpg: Release lock after close also in the compress code path.

I created gniibe/t7855 branch for this issue.
To start with, I forward-port/cherry-pick 2.2 commits to the branch:
rG39430d9f78dc: build,common,g13,sm,tools: Require GpgRT 1.56.
rGe71aca2a628d: common: New function gnupg_remove_ext.
rGe38c5f7d5873: w32:common: Take care of possible race on startup under Windows.
rG7bfd37e305c0: common,w32: Always use share mode readwrite for the keybox.

I understand that this is for 2.6.

Except for the release/unlock thing after keybox_compress I already have the other fixes in my 2.2 commits. I noticed that the gpgsm keydb lock/release stuff differes from the one for gpg: For gpg we use the keybox_lock function but that is bot used at all by gpgsm. In theory this should be unified but I fear a regression risk and thus for 2.2 we better don't touch it.

Reading the commit log message in rG6dc3846d7819: sm: Support creation of EdDSA certificates.
I created a file to keygen.

Key-Type: ECDSA
Key-Length: 1024
Key-Grip: 0286DCA85E771F64AB9FD9C89717369524D55471
Key-Usage: sign,encrypt
Hash-Algo: sha384
Serial: random
Name-DN: CN=dummy test nistp384

This is current work of mine:

Also implemented for 2.2

Will be backported after 2.2.49

Tested with VS-Desktop-3.3.90.12-Beta

2.2 test can be done with GnuPG-VS-Desktop-3.3.90.12-Beta-Standard.msi from Sep 17

Backported to 2.2 but not yes tested with 2.2

Looks good to me on gpg4win-5.0.0-beta369 @ win10

Looks good to me on gpg4win-5.0.0-beta369 @ win10.
Can't reproduce it anymore, message is S/MIME decrypted instantly:

Uses gpgme-2.0.0 with the above mentioned patches. I have seen no problems in my quick tests.

Is that really the same bug? I would be interested in seeing a more detailed report. BTW, Windows or Linux? Used standard beta installer on Windows?

In contrast to gnupg22 master did not proper show OCB compliance - not everything has yet been forward ported. But we can do so now and test master by setting GNUPG_ASSUME_COMPLIANCE=de-vs

I fixed the problem (which I identified above) in gniibe/t7759 branch. There might be other causes/problems for the particular symptom, so, I don't know the fix resolves the symptom or not, though. Anyhow, I believe that this is an improvement.

@gniibe: Now that we use the KEM API, how do we proceed with this ticket?

The problem here is that we don't have the sha-2 fingerprint in our SQL tables. Thus we would not only need to do a full table search but also parse the actual blob to compute the sha-2 fingerprint.

I have done testing using my QES certificate with all combinations of the two options.

Thank you for the report.

Similar situation could happen with gpgsm + gpg-agent, when gpg-agent is invoked by gpgsm.
(1) No gpg-agent.
(2) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(3) In gpgsm_keylist, it makes sure gpg-agent is available by GETINFO agent-check, using gpgsm_assuan_simple_command.
(4) In the server side, it tries to connect gpg-agent, invokes gpg-agent, and connect to the agent again.
(5) On Windows, it may takes time to invoke gpg-agent. And it may try to connect multiple times. Each trial may generate debug messages.
(6) When it takes too much time, the debug messages are too much. It may fill the pipe.
(7) And it blocks at log_string in my_libassuan_log_handler.
(8) ... it hangs.

Hypothetical scenario (gpgsm --server + dirmngr):
(0) It may hang when much debug messages are generated by libassuan to the pipe of --logger (diag_cb).
(1) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(2) If it's the case of standard gpgme interactions which uses gpgsm_io_event, no problem. Because the data on diag_cb is consumed well.
(3) In case of gpgsm_encrypt (or other commands), it uses gpgsm_assuan_simple_command which does not consume the data on diag_cb pipe at all.
(4) In particular, in set_recipients, gpgsm_assuan_simple_command is called by the number of recipients times.
(5) IIUC, in the server side, to handle RECIPIENT command, dirmngr is used by the call chain of:

• cmd_recipient
• gpgsm_add_to_certlist
• gpgsm_validate_chain...
• gpgsm_dirmngr_isvalid

(6) In gpgsm_dirmngr_isvalid function, libassuan is used as client side, it generates debug messages.
(7) When there are many recipients, the debug message may be big enough to fill the pipe.
(8) When pipe is filled, it blocks at log_string in my_libassuan_log_handler, waiting the data in pipe is consumed.
(9) ... it hangs.

Thanks for reporting/requesting.

Well, I will re-use this as a feature request to add this feature. Workaround is to list the key with --with-keygrip and backup the ~/.gnupg/private-keys-v1.d/<keygrip>.key files.

We decided that gpg should emit a status message for success, too.
gpgme should then look for that status message instead of only absence of error.

A quick check with passing ASSUAN_PIPE_CONNECT_DETACHED does not changed anything.

I wonder whether rA3bccb33ccd9028ff505d9979fd6c8a37393b892d which changes Assuan's waitpid function for Windows is well aligned with the my_waitpid in gpgme's assuan-support.c (which does nothing). gpgme creates a detached process in most cases but for gpgsm assuan_pipe_connect is used without the ASSUAN_PIPE_CONNECT_DETACHED flag.

Another data point is that the faulty versions use libassuan 3 with a slightly changed API. May one of the follwing chnages cause the problem?

Although in VSD 3.2.2 we get no warning when configuring S/MIME debugging wrong we then get a nice message "Configuration error" when trying to encrypt with S/MIME, instead of gpgsm hanging without any message at all:

The issue also occurs in VSD-3.3.2 and 4win-4.4.1 but not in VSD 3.1.26

The advantage of using a fingerprint for referencing a key is that there won't be any collisions in the keyid. Further this unifies the schema with an LDS (Windows) installation where DNs must anyway be unique. But take care the client needs to support this new flag. This will be the case for gnupg >= 2.5.12 (cf. T7756)

Applied the change above.

I realized that I enbugged in rG5efabec21883: gpg:ecc: Use the common function of gnupg_get_ecc_params..
It has been regression since 2.5.9.

Test on Windows by overwriting gpgtar from gpg4win-5.0.0-beta357 and also tested on Linux. Debian packages with patches are already available.