Page MenuHome GnuPG

gpg-agent: "<gcrypt> Cannot allocate memory" with 10 threads decrypting OpenPGP
Closed, ResolvedPublic


I'm setting this to needs triage as I'm not sure how important you think threading issues are. It is my opinion that as long as we don't hit system limits like max fd's or max memory the whole GnuPG System ( all components + GPGME ) should be able to handle parallel loads. And 10 operations is quite little. I can see Kleopatra starting 10 decrypts easily.

In this issue "gcrypt's Cannot allocate memory", if it is that limited the error should be handled internally or the number of currently running connections (not accepted connections) be limited.

To reproduce for me:

export GNUPGHOME=$(mktemp -d) 
gpg --yes --pinentry-mode loopback --passphrase "test" --quick-gen-key
echo Hello | gpg -er > $GNUPGHOME/pgp-enc
# unlock the key
gpg  --pinentry-mode loopback --passphrase "test" -d $GNUPGHOME/pgp-enc
./run-threaded --no-list --threads 10 --data-type 3 --repeat 10 $GNUPGHOME/pgp-enc

When I change the key to future-default it works for me.



Event Timeline

Forgot to mention. run-threaded is a new test tool in GPGME.

I can easily replicate this; it is a problem somewhere in the secure memory code of Libgcrypt.

For my case, with $GNUPGHOME/gpg-agent.conf having debug-all, I observed that rsa_decrypt failes with 'Cannot allocate memory', after debug output of 'res'.
Reading libgcrypt/cipher/rsa.c, it is line 1439, where it calls sexp_build (MPI of PLAIN into SEXP of R_PLAIN).
I think that it does indeed memory failure here.
Having "auto-expand-secmem" in gpg-agent.conf, it goes well.

Not a bug :-). I should have read my own docs before starting a long debug session. The things is that the auto expanding of the secmem area is only done for xmalloc_secure and the internal MPI allocation functions. It is not dne for any memory which is allocated with xtrymalloc becuase those properly return an error to the caller. The idea is that if the caller wants to get an error back he has also the assurance that them memory is allocated in the non-swappable memory (i.e. not in the expanded parts of the secmem).

The mitigation for your problem is to put


into gpg-agent.conf. Then the pool will we expanded for all types of allocation requests. I just ran a test with 100 threads and repeat 50 without problems. To trigger the problem earlier I also used --debug crypto, which does more allocations.

Uhm, if this option is useful why isn't it default behavior?

I would not know about this.

Agreed this looks like it should be made default behavior. This has affected many people I work with, and even with searching, this ticket never came up. I only found out about it by making a ticket myself. This issue looks like it has generated at least 3 tickets in this bug tracker, and the agent is raising memory errors during normal usage, which still smells like a bug to me.

It became common, because many people now use larger keys.
For RSA-4096, three simultaneous connections for decryption may cause the failure.
In the experimental patch of D472: Limit active connections for gpg-agent, I limit gpg-agent to accept two connections only.