As this looks good to me on gpg4win-5.0.0-beta413 @ win11, I move this to done on the gpd5x board.

Forgot to note: Setting S/MIME debug level in kleopatra via GnuPG System will write the right key to gpg.conf (if I understood it right, this was also a problem)

I think last time I didn't test the actual problem.

When --output option is used and the user uses temporary file and is ready for checking an error, that is, it's already prepared, it's redundant and useless, indeed.

With the next gpg release (2.5.14) the keyboxd has an extended fingerprint table which carries a flags column. A bit in this column can eventually be used to mark subkeys with the "R" key flag and the search funtion can be enhanced to ignore keys with that flag set. This way we can more easily lookup the actual ADSK key (with the "E" key flag) and check whether this subkey has been revoked.

@ikloecker says that Kleo already support this feature. (I didn't know that.)
So, compatibility flag to switch on/off the feature would be needed,
or this feature is not needed in GnuPG at all.

Here is my attempt to do that:

Fix applied. Thanks.

This is not a composite key specific thing despite that this is an extra challenge. The creation date is used to reconstruct a key if the public key has been lost and only the fingerprint is still available. A solution might be to test the all combinations of stored creation dates to match the fingerprint.

Werner is going to add the missing implementation so that the error will soon be gone.

Here is my idea to implement the feature:
(1) Extend struct iobuf_struct to have a field of temporary output (of int), just after real_fname.

• OUTPUTFILE: When it's 1, a file generated with real_fname original suffix removed and appended .tmp is used for the output

(2) Modify get_output_file in plaintext.c and make_outfile_name in openfile.c, so that OUTPUTFILE above is used and the field in iobuf_struct is marked.
(3) Modify proc_encrypted in mainproc.c so that rename .tmp file to the OUTPUTFILE or remove it when failure.

Let me explain the background.

@gniibe Thanks for the detailed reply. Looks like we were lucky to hit v2.5.12 when building our CI containers few weeks ago, rebuilding those now with 2.5.13 fixed the problem. Thanks again!

@onickolay The change was originally introduced for PQC stuff. And then, we applied use of KEM API (of libgcrypt) also for ordinary ECDH, so, it affected ordinary ECDH encryption (between 2.5.9 and 2.5.12).
The intention is follow the recommendation of use of KEM. IIUC, next FIPS certification will require use of KEM, possibly.

@gniibe @werner Is this change is supposed to work only for PQC stuff, or non-PQC as well, and where it is defined? As it breaks RNP tests for ordinary ECDH encryption (as it looks up for 0x40 prefix). It's not a problematic to update our code, but just want to know the reason for that.

Note that currently Kleopatra (gpg4win 5 beta) fails to delete the key, which might impact other operations. I'm currently trying to figure out, if some other bugs/quirks are a subsequent error or not.

Workaround is to use --with-keygrip and delete both <keygrip>.key files. Problem here is that one part may be on a smartcard or one part might be shared (although not allowed) with other keys.

I'd sad we keep it as it is now (unless we see a regression). The real and only correct solution is the use of a daemon to serialize access.

Still, there is a fundamental problem with keydb locking.

• It only assures no-data-corruption.
• When a process doing write access, another process reading the resource may encounter a problem (inconsistent data read), since data could be changed while accessing.
  • Currently, write access may occur with keybox compress, this means that users are not safe to invoke multiple gpg/gpgsm simultaneously (to be sure).
    • It would be: only keybox compress when users explicitly ask.
  • We could introduce a lock to read access... BUT naively adding a lock (both for read and write or read-multiple-write-one) results possible deadlock in gpgsm
    • in gpgsm, gpgsm_walk_cert_chain and gpgsm_validate_chain access the resource of keydb in a way of:
      • While it has a handle kh, by find_up routine, it may call keydb_store_cert by callback routine; The callback does write access to the resource opening another handle.
      • Currently, it works because of no lock for read access and keydb_store_cert appends data at the end.

All changes in gniibe/t7855 are pushed into master.

Implemented but not tested at all.

Lastly, pushed a change into gniibe/t7855 branch.
rGf861b2a33f96: gpg,gpgsm: Fix thinko for FP closing under no lock.

I pushed further changes into gniibe/t7855 branch.
rG2fe62809014e: gpg,gpgsm: Serialize write access to keybox/keyring to protect.

For remaining changes in 2.2, I pushed changes into gniibe/t7855 branch.
rGbd65b06b74c2: gpg,gpgsm: Don't lock recursively when KEEP_LOCK is enabled.
rG423fd047da87: kbx,gpg,gpgsm: Add FP-close method for keydb to close before unlock.
rG966258ac5f99: gpgsm: Fix delete and store certificate locking glitches.

I pushed changes into gniibe/t7855 for compressing the keybox.
rG8cc2a0e0ffee: gpg: Minor clean up for keydb_lock API.
rGe4d3c3aa2220: kbx,gpg,gpgsm: Introduce keybox_compress_when_no_other_users.
rG3e441d5b299f: kbx,gpg,gpgsm: More changes for compressing the keybox.

Then, we need to integrate following commits of 2.2 into gniibe/t7855 branch:
rG43fe9073aa81: gpg,gpgsm: Tweak the locking of the pubring.kbx
rG8491aca73cff: gpg: Revert the always locking introduced with 43fe9073aa
rGad4a5117ab1c: gpgsm: Properly release the lock when compressing a pubring.
rG7962eca3a023: gpgsm: Change delete and store certificate locking glitches.
rG22f9c4a3b3c1: gpg: Release lock after close also in the compress code path.

I created gniibe/t7855 branch for this issue.
To start with, I forward-port/cherry-pick 2.2 commits to the branch:
rG39430d9f78dc: build,common,g13,sm,tools: Require GpgRT 1.56.
rGe71aca2a628d: common: New function gnupg_remove_ext.
rGe38c5f7d5873: w32:common: Take care of possible race on startup under Windows.
rG7bfd37e305c0: common,w32: Always use share mode readwrite for the keybox.

I understand that this is for 2.6.

Except for the release/unlock thing after keybox_compress I already have the other fixes in my 2.2 commits. I noticed that the gpgsm keydb lock/release stuff differes from the one for gpg: For gpg we use the keybox_lock function but that is bot used at all by gpgsm. In theory this should be unified but I fear a regression risk and thus for 2.2 we better don't touch it.

Reading the commit log message in rG6dc3846d7819: sm: Support creation of EdDSA certificates.
I created a file to keygen.

Key-Type: ECDSA
Key-Length: 1024
Key-Grip: 0286DCA85E771F64AB9FD9C89717369524D55471
Key-Usage: sign,encrypt
Hash-Algo: sha384
Serial: random
Name-DN: CN=dummy test nistp384

This is current work of mine:

Also implemented for 2.2

Will be backported after 2.2.49

Tested with VS-Desktop-3.3.90.12-Beta

2.2 test can be done with GnuPG-VS-Desktop-3.3.90.12-Beta-Standard.msi from Sep 17

Backported to 2.2 but not yes tested with 2.2