This shows up elsewhere too:
http://forum.yubico.com/viewtopic.php?f=26&t=1171
says:
For some inexplicable reason, GnuPG cannot extract the public key from a
smartcard except during generation. That means that to use the key from
another computer, you either have to copy the public key from the original
computer's GnuPG keyring, or you need to set the URL attribute to a file
which contains the PGP public key block. Otherwise, the token is effectively
locked to a single computer, and unuseable if you happen to trash your
keyring unless you regenerate a key.
It would be nice to streamline this case.