I would say this is done.

Here is my analysis.

Yes, sorry, a typo, I corrected it.

In T7759#208677, @timegrid wrote:

Forgot to note: Setting S/MIME debug level in kleopatra via GnuPG System will write the right key to gpg.conf (if I understood it right, this was also a problem)

Looks good to me on gpg4win-5.0.0-beta413 @ win11.
The error message in debugview is gone.

Forgot to note: Setting S/MIME debug level in kleopatra via GnuPG System will write the right key to gpg.conf (if I understood it right, this was also a problem)

I think last time I didn't test the actual problem.

Great, thanks!

I considered to make the --display argument optional but that still leads to the error. Thus better do not set or send it at all. I did this now for all gpgme engines.

The API documentation of gpgme has been improved. And Kleopatra no longer tries to read the private key files of subkeys using combined algorithms (like Kyber+some curve) because (as of now) such keys are not stored on any smart cards (that are supported by GnuPG).

Looks good to me on gpg4win-5.0.0-beta369 @ win10.
Can't reproduce it anymore, message is S/MIME decrypted instantly:

Uses gpgme-2.0.0 with the above mentioned patches. I have seen no problems in my quick tests.

I fixed the problem (which I identified above) in gniibe/t7759 branch. There might be other causes/problems for the particular symptom, so, I don't know the fix resolves the symptom or not, though. Anyhow, I believe that this is an improvement.

Similar situation could happen with gpgsm + gpg-agent, when gpg-agent is invoked by gpgsm.
(1) No gpg-agent.
(2) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(3) In gpgsm_keylist, it makes sure gpg-agent is available by GETINFO agent-check, using gpgsm_assuan_simple_command.
(4) In the server side, it tries to connect gpg-agent, invokes gpg-agent, and connect to the agent again.
(5) On Windows, it may takes time to invoke gpg-agent. And it may try to connect multiple times. Each trial may generate debug messages.
(6) When it takes too much time, the debug messages are too much. It may fill the pipe.
(7) And it blocks at log_string in my_libassuan_log_handler.
(8) ... it hangs.

Hypothetical scenario (gpgsm --server + dirmngr):
(0) It may hang when much debug messages are generated by libassuan to the pipe of --logger (diag_cb).
(1) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(2) If it's the case of standard gpgme interactions which uses gpgsm_io_event, no problem. Because the data on diag_cb is consumed well.
(3) In case of gpgsm_encrypt (or other commands), it uses gpgsm_assuan_simple_command which does not consume the data on diag_cb pipe at all.
(4) In particular, in set_recipients, gpgsm_assuan_simple_command is called by the number of recipients times.
(5) IIUC, in the server side, to handle RECIPIENT command, dirmngr is used by the call chain of:

• cmd_recipient
• gpgsm_add_to_certlist
• gpgsm_validate_chain...
• gpgsm_dirmngr_isvalid

(6) In gpgsm_dirmngr_isvalid function, libassuan is used as client side, it generates debug messages.
(7) When there are many recipients, the debug message may be big enough to fill the pipe.
(8) When pipe is filled, it blocks at log_string in my_libassuan_log_handler, waiting the data in pipe is consumed.
(9) ... it hangs.

We decided that gpg should emit a status message for success, too.
gpgme should then look for that status message instead of only absence of error.

A quick check with passing ASSUAN_PIPE_CONNECT_DETACHED does not changed anything.

I wonder whether rA3bccb33ccd9028ff505d9979fd6c8a37393b892d which changes Assuan's waitpid function for Windows is well aligned with the my_waitpid in gpgme's assuan-support.c (which does nothing). gpgme creates a detached process in most cases but for gpgsm assuan_pipe_connect is used without the ASSUAN_PIPE_CONNECT_DETACHED flag.

Another data point is that the faulty versions use libassuan 3 with a slightly changed API. May one of the follwing chnages cause the problem?

Although in VSD 3.2.2 we get no warning when configuring S/MIME debugging wrong we then get a nice message "Configuration error" when trying to encrypt with S/MIME, instead of gpgsm hanging without any message at all:

The issue also occurs in VSD-3.3.2 and 4win-4.4.1 but not in VSD 3.1.26

I created a Go test program that runs several Go routines, each of which verifies a byte array loaded from a file in advance. Each go-routine is spawned with a configurable delay in milliseconds. I tested it with 100 iterations, which resulted in at least 50 parallel processes. Each verification process uses its own context, as Crio does. I didn't encounter any errors.

Here is my repository with a README containing more information: https://git.sr.ht/~kulbartsch/gpgmego-verify-load-test

Ingo tested this and it worked.

Done by: rM8caa7cc517eb: Use sysconf as a fallback mechanism in the initialization.

OK. I'll add a code for setting the fallback value in _gpgme_io_subsystem_init and use it from get_max_fds.

iirc we introduced sysconf (_SC_OPEN_MAX) for non-linux platforms and that fixed real world problems. What about getting this value at module initialization time and keep on using it as a fallback?

For issues of get_max_fds, I created a sub task, although it seems not the direct cause of this particular problem.