Allow either name or mail for smartcard keys

This is what we also do for normal keys so we should
not enforce mail for smartcards.