Here is a hypothetical application which may have similar problem.
(1) It is a multi threaded application using gpgme, forking another process (possibly, exec).
(2) One of threads invokes gpgme_new, gpgme_op_import and gpg_op_verify.
(3) When the control goes to gpgme_op_* then gpgme_io_spawn by a thread A, another thread B forks a process.
(3-1) While the thread A is polling pipe I/O, forked process holds pipe file descriptors too.
(3-2) Until the forked process exists, pipe I/O polling by the thread A continues (because pipe's other end is still active).

There is FD_CLOFORK on Solaris 11.4 as well. It is a part of POSIX-1.2024, but who knows how long until that becomes common.

I don't know if it is related to this particular case, but I found a possible race condition in _gpgme_io_pipe.
Between pipe and fcntl with FD_CLOEXEC, another thread may fork a process which keeps running.
It would be good to use pipe2 here:
https://pubs.opengroup.org/onlinepubs/9799919799/functions/pipe.html

Another possible cause is... gpgme uses closefrom in GNU C library, if available. if it doesn't work well, it would be possible invoked gpg keeps waiting its input.

Here is my observation.

This one made me curious because updating the should be UI solved, and it is incredibly dangerous to mess with that. It is super easy to get random crashes when you invalidate the UI too much. It took me ages to get that "stable enough". But also technically an appointment request is a mail. And thanks to dan (afair), KMail can sign and encrypt invitations. And at least for signed invitations they are displayed as appointment so I looked into this a bit out of curiousity.

In T4836#132493, @aheinecke wrote:

Thank you for the detailed report.

We recently had a similar problem with S/MIME Mails. T4543 I think that we can apply the same fix we did for S/MIME also for OpenPGP. So I give this high priority as I think that this can be easily fixed and is a big problem in mixed environments.

To clarify. And what I think might still not work here. Windows has the problem that it does not remove the temp directory on restart or even attempts to. So whenever we work with temporary files we need to make an effort to remove them. Because the user does not expect a decrypted file in a temporary folder to stick around forever. There are options to do that on Windows. As a last resort one could even create a registry key like we did in the uninstaller for a while to remove files which were in used after next reboot.

Actually after looking at this post here I commented on https://dev.gnupg.org/T7434#201525 Since you are already debugging and seem to be able to reproduce this somewhat more often then others. (I hope even aftger all this time? :) ) Could you try running porcmon with the generic gnupg path filter as suggested there. Will slow everything down of course, but I would love to get my fingers of such a hang like you are seeing with such a filter enabled. You can export only the filtered lines and compress it. If it is still too large or private I guess we can get you an upload space somewhere. But some unexpected situation caused by a third party software messing at the right time with our files would really fit the bill, and I already noticed that standard windows defender, in a standard windows installation messes with our lock files. Maybe you have some even more agressive software running and see such issue more often, the important thing in the procmon log would be what kind of error / access pattern occurs to figure out if we handle something not correctly, or interpret it differently.

Just as a reminder, knowledge transfer, because this is easily overlooked in testing but at least one customer would have gotten very annoyed if we had ever deployed an "Update all certificates" function which "added" new certificates. Even with the update of a single cert, we had a "funny" issue, like if you had expired certificates from anywhere and not from WKD (which old keyrings have a lot, maybe with many uids). Suddenly an update would pull in new keys which come from WKD but maybe there they all only have one UID. Because for keyservers the identifier was the fingerprint and for WKD the identifier was the userid.
Or even worse, you explicitly threw out the OpenPGP keys from WKD because you wanted to use only S/MIME, then such a function may not search on any OpenPGP Sources.
When I worked at Kleopatra we didn't want such a feature in GnuPG. Our strategy was to update keys when they are used, about to be used or close to expiry. The whole locate-external-key thing.
I think the feature we had to update in the certificate details is good. But i recommend especially keeping the S/MIME / OpenPGP difference in mind. I would also call it "Search updated certificates" with a tooltip that it might also find "new" certificates for the user. And then an option to disable this either for S/MIME or for OpenPGP.

In T7656#201519, @TobiasFella wrote:

Do I understand correctly that this bug is then automatically done/fixed?

I do not think that this is the only place where such an issue occurs. Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

The more I think of this, the more likely this appears to me as the source for all that random startup weirdness of GnuPG. Say you are on a large keyring and on a train, then that keyring is first passed through your enterprise malware protection for scanning or something like that. Then it works again until some metric, hash or something else changes.

My recommendation would at this point be to use procmon with a file filter for just "If path contains gnupg then include" I mean maybe go only for the locking dirs but this way you will not only see what the GnuPG processes are doing but what everyone on the system is doing to the locks. So you will see when my old friends, third party security software might interfere.
For example: You will see on a default Windows which files are checked through telemetry. And here in this example you see directly that the Microsoft Malware Protection Engine is accessing the agents socket.

Yes. If gpgconf could read that version directly from kleopatra it would be even better. Bit in cases of early crashes this might be sub-optimal; thus I will tell gpgconf to get some additional version info from an installed versioninfo.txt file (which gpg4win creates). Thanks.

@gnibee Thanks! I noticed this too but wasn't familiar with the test suite, and didn't see an ASAN error so assumed nothing harmful. This explanation explains it.

Is this what you had in mind @werner:

Note: The Kleopatra in upcoming versions of Gpg4win 5 will have AboutData::version set to gpg4win-5.0.0 (or gpg4win-5.0.0-beta190 for beta versions). See T7666: Kleopatra: Rework versioning.