Currently, the text which is set by gpg-agent's SETKEYDESC is not only a description for a key itself, but also includes context of an operation with the key.

Because of that, when a key is on card/token, the text is just ignored (not used at all), although a part of description (a description for a key itself) is useful for a user.

In g10/passphrase.c, we have a function gpg_format_keydesc, which is used to generate a text to be used for SETKEYDESC.

Perhaps, it would be good to have two (or more) separate texts, one for the key description itself, and another for context (operation) for the key.