Here is my idea to implement the feature:
(1) Extend struct iobuf_struct to have a field of temporary output (of int), just after real_fname.

• OUTPUTFILE: When it's 1, a file generated with real_fname original suffix removed and appended .tmp is used for the output

(2) Modify get_output_file in plaintext.c and make_outfile_name in openfile.c, so that OUTPUTFILE above is used and the field in iobuf_struct is marked.
(3) Modify proc_encrypted in mainproc.c so that rename .tmp file to the OUTPUTFILE or remove it when failure.

Let me explain the background.

@gniibe Thanks for the detailed reply. Looks like we were lucky to hit v2.5.12 when building our CI containers few weeks ago, rebuilding those now with 2.5.13 fixed the problem. Thanks again!

@onickolay The change was originally introduced for PQC stuff. And then, we applied use of KEM API (of libgcrypt) also for ordinary ECDH, so, it affected ordinary ECDH encryption (between 2.5.9 and 2.5.12).
The intention is follow the recommendation of use of KEM. IIUC, next FIPS certification will require use of KEM, possibly.

@gniibe @werner Is this change is supposed to work only for PQC stuff, or non-PQC as well, and where it is defined? As it breaks RNP tests for ordinary ECDH encryption (as it looks up for 0x40 prefix). It's not a problematic to update our code, but just want to know the reason for that.

Note that currently Kleopatra (gpg4win 5 beta) fails to delete the key, which might impact other operations. I'm currently trying to figure out, if some other bugs/quirks are a subsequent error or not.

Workaround is to use --with-keygrip and delete both <keygrip>.key files. Problem here is that one part may be on a smartcard or one part might be shared (although not allowed) with other keys.

I'd sad we keep it as it is now (unless we see a regression). The real and only correct solution is the use of a daemon to serialize access.

Still, there is a fundamental problem with keydb locking.

• It only assures no-data-corruption.
• When a process doing write access, another process reading the resource may encounter a problem (inconsistent data read), since data could be changed while accessing.
  • Currently, write access may occur with keybox compress, this means that users are not safe to invoke multiple gpg/gpgsm simultaneously (to be sure).
    • It would be: only keybox compress when users explicitly ask.
  • We could introduce a lock to read access... BUT naively adding a lock (both for read and write or read-multiple-write-one) results possible deadlock in gpgsm
    • in gpgsm, gpgsm_walk_cert_chain and gpgsm_validate_chain access the resource of keydb in a way of:
      • While it has a handle kh, by find_up routine, it may call keydb_store_cert by callback routine; The callback does write access to the resource opening another handle.
      • Currently, it works because of no lock for read access and keydb_store_cert appends data at the end.

All changes in gniibe/t7855 are pushed into master.

Implemented but not tested at all.

Lastly, pushed a change into gniibe/t7855 branch.
rGf861b2a33f96: gpg,gpgsm: Fix thinko for FP closing under no lock.

I pushed further changes into gniibe/t7855 branch.
rG2fe62809014e: gpg,gpgsm: Serialize write access to keybox/keyring to protect.

For remaining changes in 2.2, I pushed changes into gniibe/t7855 branch.
rGbd65b06b74c2: gpg,gpgsm: Don't lock recursively when KEEP_LOCK is enabled.
rG423fd047da87: kbx,gpg,gpgsm: Add FP-close method for keydb to close before unlock.
rG966258ac5f99: gpgsm: Fix delete and store certificate locking glitches.

I pushed changes into gniibe/t7855 for compressing the keybox.
rG8cc2a0e0ffee: gpg: Minor clean up for keydb_lock API.
rGe4d3c3aa2220: kbx,gpg,gpgsm: Introduce keybox_compress_when_no_other_users.
rG3e441d5b299f: kbx,gpg,gpgsm: More changes for compressing the keybox.

Then, we need to integrate following commits of 2.2 into gniibe/t7855 branch:
rG43fe9073aa81: gpg,gpgsm: Tweak the locking of the pubring.kbx
rG8491aca73cff: gpg: Revert the always locking introduced with 43fe9073aa
rGad4a5117ab1c: gpgsm: Properly release the lock when compressing a pubring.
rG7962eca3a023: gpgsm: Change delete and store certificate locking glitches.
rG22f9c4a3b3c1: gpg: Release lock after close also in the compress code path.

I created gniibe/t7855 branch for this issue.
To start with, I forward-port/cherry-pick 2.2 commits to the branch:
rG39430d9f78dc: build,common,g13,sm,tools: Require GpgRT 1.56.
rGe71aca2a628d: common: New function gnupg_remove_ext.
rGe38c5f7d5873: w32:common: Take care of possible race on startup under Windows.
rG7bfd37e305c0: common,w32: Always use share mode readwrite for the keybox.

I understand that this is for 2.6.

Except for the release/unlock thing after keybox_compress I already have the other fixes in my 2.2 commits. I noticed that the gpgsm keydb lock/release stuff differes from the one for gpg: For gpg we use the keybox_lock function but that is bot used at all by gpgsm. In theory this should be unified but I fear a regression risk and thus for 2.2 we better don't touch it.

Reading the commit log message in rG6dc3846d7819: sm: Support creation of EdDSA certificates.
I created a file to keygen.

Key-Type: ECDSA
Key-Length: 1024
Key-Grip: 0286DCA85E771F64AB9FD9C89717369524D55471
Key-Usage: sign,encrypt
Hash-Algo: sha384
Serial: random
Name-DN: CN=dummy test nistp384

This is current work of mine:

Also implemented for 2.2

Will be backported after 2.2.49

Tested with VS-Desktop-3.3.90.12-Beta

2.2 test can be done with GnuPG-VS-Desktop-3.3.90.12-Beta-Standard.msi from Sep 17

Backported to 2.2 but not yes tested with 2.2

Looks good to me on gpg4win-5.0.0-beta369 @ win10

Looks good to me on gpg4win-5.0.0-beta369 @ win10.
Can't reproduce it anymore, message is S/MIME decrypted instantly:

Uses gpgme-2.0.0 with the above mentioned patches. I have seen no problems in my quick tests.

Is that really the same bug? I would be interested in seeing a more detailed report. BTW, Windows or Linux? Used standard beta installer on Windows?

In contrast to gnupg22 master did not proper show OCB compliance - not everything has yet been forward ported. But we can do so now and test master by setting GNUPG_ASSUME_COMPLIANCE=de-vs

I fixed the problem (which I identified above) in gniibe/t7759 branch. There might be other causes/problems for the particular symptom, so, I don't know the fix resolves the symptom or not, though. Anyhow, I believe that this is an improvement.

@gniibe: Now that we use the KEM API, how do we proceed with this ticket?

The problem here is that we don't have the sha-2 fingerprint in our SQL tables. Thus we would not only need to do a full table search but also parse the actual blob to compute the sha-2 fingerprint.

I have done testing using my QES certificate with all combinations of the two options.

Thank you for the report.

Similar situation could happen with gpgsm + gpg-agent, when gpg-agent is invoked by gpgsm.
(1) No gpg-agent.
(2) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(3) In gpgsm_keylist, it makes sure gpg-agent is available by GETINFO agent-check, using gpgsm_assuan_simple_command.
(4) In the server side, it tries to connect gpg-agent, invokes gpg-agent, and connect to the agent again.
(5) On Windows, it may takes time to invoke gpg-agent. And it may try to connect multiple times. Each trial may generate debug messages.
(6) When it takes too much time, the debug messages are too much. It may fill the pipe.
(7) And it blocks at log_string in my_libassuan_log_handler.
(8) ... it hangs.

Hypothetical scenario (gpgsm --server + dirmngr):
(0) It may hang when much debug messages are generated by libassuan to the pipe of --logger (diag_cb).
(1) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(2) If it's the case of standard gpgme interactions which uses gpgsm_io_event, no problem. Because the data on diag_cb is consumed well.
(3) In case of gpgsm_encrypt (or other commands), it uses gpgsm_assuan_simple_command which does not consume the data on diag_cb pipe at all.
(4) In particular, in set_recipients, gpgsm_assuan_simple_command is called by the number of recipients times.
(5) IIUC, in the server side, to handle RECIPIENT command, dirmngr is used by the call chain of:

• cmd_recipient
• gpgsm_add_to_certlist
• gpgsm_validate_chain...
• gpgsm_dirmngr_isvalid

(6) In gpgsm_dirmngr_isvalid function, libassuan is used as client side, it generates debug messages.
(7) When there are many recipients, the debug message may be big enough to fill the pipe.
(8) When pipe is filled, it blocks at log_string in my_libassuan_log_handler, waiting the data in pipe is consumed.
(9) ... it hangs.

Thanks for reporting/requesting.