Completed working base repository with developer workflows for watching files and rebuilding/retesting:
https://github.com/anthumchris/node-addon

• --experimental-addon-modules stability for javascript ESM import syntax

Thanks, I'll start here and see how it was done with JS for the browser: https://dev.gnupg.org/source/gpgme/browse/master/lang/js/

Note that we have moved almost all bindings out of gpgme into separate repos. I suggest to develop such bindings externally. And you'll have to find external resources to learn how to create nodejs bindings for gpgme.

I would say this is done.

Here is my analysis.

Yes, sorry, a typo, I corrected it.

In T7759#208677, @timegrid wrote:

Forgot to note: Setting S/MIME debug level in kleopatra via GnuPG System will write the right key to gpg.conf (if I understood it right, this was also a problem)

Looks good to me on gpg4win-5.0.0-beta413 @ win11.
The error message in debugview is gone.

Forgot to note: Setting S/MIME debug level in kleopatra via GnuPG System will write the right key to gpgsm.conf (if I understood it right, this was also a problem)

I think last time I didn't test the actual problem.

Great, thanks!

I considered to make the --display argument optional but that still leads to the error. Thus better do not set or send it at all. I did this now for all gpgme engines.

The API documentation of gpgme has been improved. And Kleopatra no longer tries to read the private key files of subkeys using combined algorithms (like Kyber+some curve) because (as of now) such keys are not stored on any smart cards (that are supported by GnuPG).

Looks good to me on gpg4win-5.0.0-beta369 @ win10.
Can't reproduce it anymore, message is S/MIME decrypted instantly:

Uses gpgme-2.0.0 with the above mentioned patches. I have seen no problems in my quick tests.

I fixed the problem (which I identified above) in gniibe/t7759 branch. There might be other causes/problems for the particular symptom, so, I don't know the fix resolves the symptom or not, though. Anyhow, I believe that this is an improvement.

Similar situation could happen with gpgsm + gpg-agent, when gpg-agent is invoked by gpgsm.
(1) No gpg-agent.
(2) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(3) In gpgsm_keylist, it makes sure gpg-agent is available by GETINFO agent-check, using gpgsm_assuan_simple_command.
(4) In the server side, it tries to connect gpg-agent, invokes gpg-agent, and connect to the agent again.
(5) On Windows, it may takes time to invoke gpg-agent. And it may try to connect multiple times. Each trial may generate debug messages.
(6) When it takes too much time, the debug messages are too much. It may fill the pipe.
(7) And it blocks at log_string in my_libassuan_log_handler.
(8) ... it hangs.

Hypothetical scenario (gpgsm --server + dirmngr):
(0) It may hang when much debug messages are generated by libassuan to the pipe of --logger (diag_cb).
(1) In gpgme, by engine-gpgsm, gpgsm is invoked with --logger.
(2) If it's the case of standard gpgme interactions which uses gpgsm_io_event, no problem. Because the data on diag_cb is consumed well.
(3) In case of gpgsm_encrypt (or other commands), it uses gpgsm_assuan_simple_command which does not consume the data on diag_cb pipe at all.
(4) In particular, in set_recipients, gpgsm_assuan_simple_command is called by the number of recipients times.
(5) IIUC, in the server side, to handle RECIPIENT command, dirmngr is used by the call chain of:

• cmd_recipient
• gpgsm_add_to_certlist
• gpgsm_validate_chain...
• gpgsm_dirmngr_isvalid

(6) In gpgsm_dirmngr_isvalid function, libassuan is used as client side, it generates debug messages.
(7) When there are many recipients, the debug message may be big enough to fill the pipe.
(8) When pipe is filled, it blocks at log_string in my_libassuan_log_handler, waiting the data in pipe is consumed.
(9) ... it hangs.

We decided that gpg should emit a status message for success, too.
gpgme should then look for that status message instead of only absence of error.

A quick check with passing ASSUAN_PIPE_CONNECT_DETACHED does not changed anything.

I wonder whether rA3bccb33ccd9028ff505d9979fd6c8a37393b892d which changes Assuan's waitpid function for Windows is well aligned with the my_waitpid in gpgme's assuan-support.c (which does nothing). gpgme creates a detached process in most cases but for gpgsm assuan_pipe_connect is used without the ASSUAN_PIPE_CONNECT_DETACHED flag.

Another data point is that the faulty versions use libassuan 3 with a slightly changed API. May one of the follwing chnages cause the problem?

Although in VSD 3.2.2 we get no warning when configuring S/MIME debugging wrong we then get a nice message "Configuration error" when trying to encrypt with S/MIME, instead of gpgsm hanging without any message at all:

The issue also occurs in VSD-3.3.2 and 4win-4.4.1 but not in VSD 3.1.26

I created a Go test program that runs several Go routines, each of which verifies a byte array loaded from a file in advance. Each go-routine is spawned with a configurable delay in milliseconds. I tested it with 100 iterations, which resulted in at least 50 parallel processes. Each verification process uses its own context, as Crio does. I didn't encounter any errors.

Here is my repository with a README containing more information: https://git.sr.ht/~kulbartsch/gpgmego-verify-load-test

Ingo tested this and it worked.

Done by: rM8caa7cc517eb: Use sysconf as a fallback mechanism in the initialization.

OK. I'll add a code for setting the fallback value in _gpgme_io_subsystem_init and use it from get_max_fds.

iirc we introduced sysconf (_SC_OPEN_MAX) for non-linux platforms and that fixed real world problems. What about getting this value at module initialization time and keep on using it as a fallback?

For issues of get_max_fds, I created a sub task, although it seems not the direct cause of this particular problem.

I test following test program (gcc -o t-gmf t-gmf.c) on Debian machine of S390x.

In the log, we can observe duplicated lines generated by
https://dev.gnupg.org/source/gpgme/browse/master/src/posix-io.c$545
Example is like:

2025-05-19 20:16:35 gpgme[21970.55d7]   _gpgme_io_spawn: check: fd[0] = 0x1c -> 0x1
2025-05-19 20:16:35 gpgme[21970.55d7]   _gpgme_io_spawn: check: fd[0] = 0x1c -> 0x1

In practice, calling gpgme_get_key() will often pick up most changes because GPGME asks the underlying GPG agent daemon, which may re-read the keyring. That gives the impression that a long-lived context automatically reflects live updates. However, as aheinecke noted, some updates can still go unnoticed in a single gpgme_ctx_t, so it isn’t a strictly frozen snapshot nor a perfectly live view—behaviors are mixed.

In T7620#201528, @aheinecke wrote:

Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

In T7673#201735, @werner wrote:

We will keep the 1.24 branch for bug fixing. Do you really have a problem with gpgme 2.0? Ist that due to factoring out the c++ and qt bindings? That was actually a long standing request from the KDE community so that they can use their cmake stuff.

We will keep the 1.24 branch for bug fixing. Do you really have a problem with gpgme 2.0? Ist that due to factoring out the c++ and qt bindings? That was actually a long standing request from the KDE community so that they can use their cmake stuff.

I'm wondering whether gpgme v1 will still be supported? Or it will be abandoned from now on.

No other processes that I have seen. However please see this last update by Kulbarsch. https://github.com/cri-o/cri-o/issues/8906#issuecomment-2936351035

@sj98ta
Does cri-o invokes processes (other than the ones of gpgme) by its threads?

I am not sure that I am clear on what you are asking. I am not an expert on cri-o, but is does seem that there are multiple processes (threads), which all call gpgme_op_verify There are also calls to gpgme_op_import

@sj98ta Please let us know if cri-o invokes other processes (except the ones by gpgme) or not.
If cri-o invokes other processes (by other threads), my theory matters; With the interference by other processes holding pipe file descriptors, gpgme keeps polling pipe file descriptors.

I have now seen instances where 1, 2, or 3 processes hang.

Re: pipe2: In gpgme_io_pipe we set FD_CLOEXEC only for one end of the pipe. Thus simply using pipe2 would change the behaviour.

Here is a hypothetical application which may have similar problem.
(1) It is a multi threaded application using gpgme, forking another process (possibly, exec).
(2) One of threads invokes gpgme_new, gpgme_op_import and gpg_op_verify.
(3) When the control goes to gpgme_op_* then gpgme_io_spawn by a thread A, another thread B forks a process.
(3-1) While the thread A is polling pipe I/O, forked process holds pipe file descriptors too.
(3-2) Until the forked process exists, pipe I/O polling by the thread A continues (because pipe's other end is still active).

There is FD_CLOFORK on Solaris 11.4 as well. It is a part of POSIX-1.2024, but who knows how long until that becomes common.

I don't know if it is related to this particular case, but I found a possible race condition in _gpgme_io_pipe.
Between pipe and fcntl with FD_CLOEXEC, another thread may fork a process which keeps running.
It would be good to use pipe2 here:
https://pubs.opengroup.org/onlinepubs/9799919799/functions/pipe.html

Another possible cause is... gpgme uses closefrom in GNU C library, if available. if it doesn't work well, it would be possible invoked gpg keeps waiting its input.

Here is my observation.

I do not think that this is the only place where such an issue occurs. Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

looks good to me on gpg4win-4.4.1-beta59@win10

Checked with Gpg4win-4.4.1-beta59, too, which contains gpgme 1.24.3. Works!