Your problem is apparently not an issue of upstream development of GnuPG; It is your setup script (agent.sh?) which specifies /dev/shm/SOMETHING.
Standard GnuPG never does that. We have no idea about use of /dev/shm/SOMETHING.

fwiw. Your patch is beautiful in which it follows our coding style and debug output. I'm confident that we will accept it but currently I have to read up on Job's a bit.

That is a very interesting problem that we did not have on our radar.

When bogus entry is "", the error is GPG_ERR_NO_PASSPHRASE, and user cannot input the passphrase.

Confirmed that manually created entry in gnome-keyring-daemon causes trouble.

Since there is --mode=normal option, it should be --mode=ssh.

I've developed a simple patch that sets the CREATE_BREAKAWAY_FROM_JOB flag when creating a new background process. This flag requires a special permission on the job object, which is tested first. This means that the patch only works if the parent process sets JOB_OBJECT_LIMIT_BREAKAWAY_OK on the job object, otherwise the behavior should be as without the patch.

Right. We won't change that though. Sorry.

FWIW, the canonical way to make sure that gpg-agent has been started is to run

You're very welcome. In my instance, this is "resolved" - I now get the prompt I realised I needed so to me this bug could be considered closed or wontfix, but I'll leave you to do with it as you please.

Basically, you are right. In addition, gpg-agent asks scdaemon about list of card/token.

OK - so if an entry is not required in sshcontrol for a smart-card key - is the private key stub sufficiently detailed enough for the agent to realise that it can ask for that card to be inserted for an ssh connection?

sshcontrol entry is required for non-smartcard keys, but not for keys on smartcard. This is intentional. For gpg-agent and current format, it is only the information for gpg-agent to know if a key is for SSH or not.

Also - going back to sshcontrol - with an ssh key added to the agent with ssh-add, an entry in sshcontrol is required - but not for a key on a smartcard. Is that intentional, or just a byproduct of the smartcard diversion that happens?

Oh, wow - yes, adding to sshcontrol brings up the prompt - I do however need to stop the agent from being restarted on insertion for it to subsequently ask for the unlock.

I see your point. You are right. For SSH access, it just fails without asking insertion. It's not Windows specific.
I checked the change of history of gpg-agent, but I cannot find prompting insertion was supported.
So, I don't thin this is a regression.

Yes, it's running. I have a scheduled task that spawns a vbscript to ensure that gpg-agent is started on login, and restarts it on insertion of a card (specifically for two reasons: windows ssh clients don't typically start agents automatically, and windows can cause gpg-agent to get a but upset after a card is removed and re-inserted. Edit: although, I think that latter reason might be resolved now... I haven't investigated deeply. more info here and here).

Thanks for your information.
Hum, you are using gpg-agent for SSH access.

When no card is inserted, usage of an ssh client simply fails to request insertion of the card for the stub keys present in ~/.gnupg/.

Asked to raise the priority on this. The quality bar issue is T2103

Please let us know the version of GnuPG, the output of gpg --card-status when inserted, and how gpg is not working well, etc.

How scdaemon responds when there is no card available?

In Wald someone reports that this also appears to happen when decrypting. https://wald.intevation.org/forum/message.php?msg_id=6377 Probably run-threaded will help to flush this out.

It became common, because many people now use larger keys.
For RSA-4096, three simultaneous connections for decryption may cause the failure.
In the experimental patch of D472: Limit active connections for gpg-agent, I limit gpg-agent to accept two connections only.

Agreed this looks like it should be made default behavior. This has affected many people I work with, and even with searching, this ticket never came up. I only found out about it by making a ticket myself. This issue looks like it has generated at least 3 tickets in this bug tracker, and the agent is raising memory errors during normal usage, which still smells like a bug to me.

Uhm, if this option is useful why isn't it default behavior?

The --auto-expand-secmem option is available in 2.2. and master for quite some time. It works if libgcrypt 1.8.2 or newer is used.

Not a bug :-). I should have read my own docs before starting a long debug session. The things is that the auto expanding of the secmem area is only done for xmalloc_secure and the internal MPI allocation functions. It is not dne for any memory which is allocated with xtrymalloc becuase those properly return an error to the caller. The idea is that if the caller wants to get an error back he has also the assurance that them memory is allocated in the non-swappable memory (i.e. not in the expanded parts of the secmem).

For my case, with $GNUPGHOME/gpg-agent.conf having debug-all, I observed that rsa_decrypt failes with 'Cannot allocate memory', after debug output of 'res'.
Reading libgcrypt/cipher/rsa.c, it is line 1439, where it calls sexp_build (MPI of PLAIN into SEXP of R_PLAIN).
I think that it does indeed memory failure here.
Having "auto-expand-secmem" in gpg-agent.conf, it goes well.

I can easily replicate this; it is a problem somewhere in the secure memory code of Libgcrypt.

..... And now after looking into this a bit deeper after attempting to build gpg-agent for windows, it appears that this is a bit deeper than the logic above (which is actually sound, when I read it for the second time)

Hmmm

You seem to accept it. So Normal Prio and assigned to you :-p

Just as a note: I think the main selling point of GnuPG is that its stable. We care about backwards compatibility and we (are || want to be) rock solid. Even if there is a rare race. With millions of installations, that race will happen regularly. So I really would like us to get all this fixed without losing to much performance by locking to much.

Happens though. With the test invocation above there is only one key in the keyring.

1.9.0-beta68

Well, it should not happen if you always use the same key.

There is indeed a race condition between the passphrase cache and the pinentry invocation. There is even a comment on this somewhere in the code. The problem is that we would need to lock almost everything to avoid this rare condition.

Which Libgcrypt version?

Forgot to mention. run-threaded is a new test tool in GPGME.

Marking this as resolved as it was forgotten in the testing state.

We had this idea to have a label: or similar item in the extended-key-format which is displayed in addition to the other info. The user can then use an editor to put whatever she likes into this field.

there should be clearer labelling of smartcards so that users can tell them apart more easily

I think it's good to close this as "resolved", since many fixes have been done, and I don't have remaining issue.
@wiz Please open another ticket for your next try.

What are we going to do with this report? The last comment is 6 months old; can we change from testing to resolved or do we need to wait for a gpgme release?

This entry was created based on the conversation at #gnupg channel.
I can't reproduce keep hanging.
I confirmed that pinentry vanished (perhaps, because of timeout).

gpg-agent has a pinentry caling timeout - doesn't that trigger?
In any case we agreed that Debian takes care of systemd support because that is not an upstream supported configuration.

Thanks. I added all standard names to that list.

@werner I was hoping to make a modified gpg-agent build that would let me walk through what's going on after the nonce is sent but it looks like the gpg4win process only takes in a package of pre-built gpg binaries which rules that out. As far as I can figure out, after the nonce is read and accepted, libassuan creates a stream object out of the socket and then finding nothing in the stream terminates the ssh handler. We send the actual client request immediately after the nonce but in a separate call to send() so I now wonder if by not having anything read in at the same time as the nonce gpg-agent or libassuan thinks that it's a 0-length stream.