For some reason (say, a bug in gnome-keyring-daemon), it happens that invalid entry is recorded in gnome-keyring-daemon.
Once it's there, gpg-agent keep failing to unprotect key using invalid cached passphrase.
For such a case, gpg-agent should ask pinentry to clear the cache, and pinentry should ask the external cache service (libsecret) to clear the cache.
Or else, creating invalid entry (say, using gpg-keyring-daemon's vulnerability) would be a kind of DoS for use of private key.