Hi.
The man pages for both gpg and gpg2 (confirmed in 1.4.12 and
2.0.19) read:
--delete-secret-key name Remove key from the secret and public keyring. In batch mode the key must be specified by fingerprint.
As the flag suggests, the command removes the secret key but does
not remove the key from the public keyring. It should read something like:
--delete-secret-key name Remove key from the secret keyring. In batch mode the key must be specified by fingerprint.