Shadowed key includes serial number of the smartcard, so that GnuPG can ask a
user about the specific card when signature/decryption will be asked (but not
yet the card is ready).
This is better to improve. (1) some people have same a practice to put private
key on multiple smartcards (say, for backup). (2) a user wants to migrate to
new card/token. (3) Most users don't care/know about serial number of smartcard
which is recorded in the shadowed key.