We can use gpg-agent as ssh-agent. This usually works well, but it comes with some caveats which are confusing.

For instance, let's say I have a remote host with multiple keys configured in ~/.ssh/authorized_keys. These keys are on different smartcards, and they've both been used in the past, so the stubs have been generated (at least this is my understanding). For the record, I'm using Debian 9, which stock scdaemon (no pcsc-lite or external ccid driver), but I've seen the same behavior elsewhere, such as on OpenBSD.

If I have one of the accepted cards inserted when I issue the command to log in over SSH, it will prompt for my card fine and will not prompt for other smartcards.

However, if I don't, it gets more confusing. The order that the cards are prompted for naturally correspond to the order in which the corresponding SSH public keys are declared in ~/.ssh/authorized_keys. Let's say I have two keys, let's call them "key no. 1" and "key no. 2". Key no. 1 is prompted for first. If I insert the corresponding card and select "OK", I'll be prompted for my PIN as normal.

However, if I insert the other one, and select 'OK', it continues to prompt for the card on which key no. 1 is stored. Instead, one would select cancel and it would move on to the next card on which key no. 2 is stored.

In my opinion this is quite confusing for users. It is unlikely they will memorise the exact ID of the card that is being prompted for, so they will think that something is wrong and may be afraid to select 'cancel'. I think there should be clearer labelling of smartcards so that users can tell them apart more easily. I also think that pinentry could respond to this case in a more intuitive manner; prompting for PINs for smartcards is a different workflow to prompting for passphrases for local keys, so it should be possible to make it clearer to users that pressing 'cancel' might move along to the next card.