When the current implementation does operations like decrypt and
encrypt with a passphrase provided as a function argument, it
temporarily changes the pinentry mode and sets up a passphrase
callback.

After finishing the operation, the pinentry mode is reset to the
previous state, but if there was no callback function previously, the
passphrase callback is not reset. This keeps a reference to the
passphrase function active, which in turn has a reference to the
passphrase itself. This may keep the passphrase in memory
unexpectedly.