Shall we merge this with T2196 ? BTW, I have some unpushed commit and a test installer.

Except for the release/unlock thing after keybox_compress I already have the other fixes in my 2.2 commits. I noticed that the gpgsm keydb lock/release stuff differes from the one for gpg: For gpg we use the keybox_lock function but that is bot used at all by gpgsm. In theory this should be unified but I fear a regression risk and thus for 2.2 we better don't touch it.

Let us keep this ticket for things in gnupg26 and later with it additional support of keyboxd. And use the old ticket for 2.2 which has anyway a different code layout in both keydb.c modules.

I understand that this is for 2.6.

I had misunderstood that the way of gpg and the way of gpgsm were same for the locking/unlocking. No, it's different.

For gpg, it locks (by dotlock) pubring.kbx.lock both for read access and write access.
For gpgsm, it only locks only for write access.

I created gniibe/t7855 branch for this issue.
To start with, I forward-port/cherry-pick 2.2 commits to the branch:
rG39430d9f78dc: build,common,g13,sm,tools: Require GpgRT 1.56.
rGe71aca2a628d: common: New function gnupg_remove_ext.
rGe38c5f7d5873: w32:common: Take care of possible race on startup under Windows.
rG7bfd37e305c0: common,w32: Always use share mode readwrite for the keybox.

Then, we need to integrate following commits of 2.2 into gniibe/t7855 branch:
rG43fe9073aa81: gpg,gpgsm: Tweak the locking of the pubring.kbx
rG8491aca73cff: gpg: Revert the always locking introduced with 43fe9073aa
rGad4a5117ab1c: gpgsm: Properly release the lock when compressing a pubring.
rG7962eca3a023: gpgsm: Change delete and store certificate locking glitches.
rG22f9c4a3b3c1: gpg: Release lock after close also in the compress code path.

I pushed changes into gniibe/t7855 for compressing the keybox.
rG8cc2a0e0ffee: gpg: Minor clean up for keydb_lock API.
rGe4d3c3aa2220: kbx,gpg,gpgsm: Introduce keybox_compress_when_no_other_users.
rG3e441d5b299f: kbx,gpg,gpgsm: More changes for compressing the keybox.

So, remaining are:
rG43fe9073aa81: gpg,gpgsm: Tweak the locking of the pubring.kbx
rG8491aca73cff: gpg: Revert the always locking introduced with 43fe9073aa
rG7962eca3a023: gpgsm: Change delete and store certificate locking glitches.

For remaining changes in 2.2, I pushed changes into gniibe/t7855 branch.
rGbd65b06b74c2: gpg,gpgsm: Don't lock recursively when KEEP_LOCK is enabled.
rG423fd047da87: kbx,gpg,gpgsm: Add FP-close method for keydb to close before unlock.
rG966258ac5f99: gpgsm: Fix delete and store certificate locking glitches.

I pushed further changes into gniibe/t7855 branch.
rG2fe62809014e: gpg,gpgsm: Serialize write access to keybox/keyring to protect.

The way of keydb access is serialized for write access, so that write access always results consistent data (never corrupts data).
(Note that there is no guarantee that for read access with consistent data; when it's under write access, it may see inconsistent data.)

Before the changes of this commit, the places for write access is somehow "locked", but not correctly thoroughly;
The access pattern should be: new+open, lock, search (locating the offset), write, and close+unlock+release

If the search (locating the offset) is not locked, it may result corrupted data, because it may call keybox_compress between search and write, for example.

Lastly, pushed a change into gniibe/t7855 branch.
rGf861b2a33f96: gpg,gpgsm: Fix thinko for FP closing under no lock.

The change:
rG2fe62809014e: gpg,gpgsm: Serialize write access to keybox/keyring to protect.
This introduces behavior change; When multiple gpg --key-edit are invoked, only one get a lock, others have to wait the lock.
But this means that: before the changes, it might result corrupted pubring.kbx/pubring.gpg.

Still, there is a fundamental problem with keydb locking.

• It only assures no-data-corruption.
• When a process doing write access, another process reading the resource may encounter a problem (inconsistent data read), since data could be changed while accessing.
  • Currently, write access may occur with keybox compress, this means that users are not safe to invoke multiple gpg/gpgsm simultaneously (to be sure).
    • It would be: only keybox compress when users explicitly ask.
  • We could introduce a lock to read access... BUT naively adding a lock (both for read and write or read-multiple-write-one) results possible deadlock in gpgsm
    • in gpgsm, gpgsm_walk_cert_chain and gpgsm_validate_chain access the resource of keydb in a way of:
      • While it has a handle kh, by find_up routine, it may call keydb_store_cert by callback routine; The callback does write access to the resource opening another handle.
      • Currently, it works because of no lock for read access and keydb_store_cert appends data at the end.

I'd sad we keep it as it is now (unless we see a regression). The real and only correct solution is the use of a daemon to serialize access.