For OpenPGP there already exists an Address book integration to override / set the keys used for a specific contact.

For S/MIME this would also make sense. As we might even support mixed encryption in the future it should be implemented in a way that both S/MIME and OpenPGP Keys can be set for a contact.