Fixed in 2.5.8.

In case of problems with token based cv25519 key, please update to 2.5.8.

The actual project we had in mind for this was more or less canceled and thus I re-prioritize this task.

In Kleopatra we explicitly trigger a re-reading of the smart card after each operation involving a smart card to ensure that Kleopatra doesn't show wrong information. There's so much that can go wrong with physical smart cards that this is the only way to make sure you don't tell the user lies. I think gpg --edit-card also re-reads the smart card after each operation.

There is no bug in the contexts and there's nothing to document anywhere. If anything then it's a bug in gpg's generate command or a more general issue (in gpg-agent) with keeping track of the storage location of private keys as I have already explained in T7620#200613. I'm removing the gpgme tag because there's nothing wrong in gpgme and there's nothing we can do in gpgme. It needs to be addressed in gnupg.

In practice, calling gpgme_get_key() will often pick up most changes because GPGME asks the underlying GPG agent daemon, which may re-read the keyring. That gives the impression that a long-lived context automatically reflects live updates. However, as aheinecke noted, some updates can still go unnoticed in a single gpgme_ctx_t, so it isn’t a strictly frozen snapshot nor a perfectly live view—behaviors are mixed.

In T7620#201528, @aheinecke wrote:

Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

The problem was: In scdaemon, PKSIGN with OPENPGP.3 didn't work well for Ed25519 (done by do_auth function in app-openpgp.c), when --hash=sha512 (not SHA1).

I located the bug in scdaemon.

Pushed the change: rG16ee68259d1d: gpg,regexp: Use -DREGEXP_PREFIX=gnupg_.

I do not think that this is the only place where such an issue occurs. Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

The more I think of this, the more likely this appears to me as the source for all that random startup weirdness of GnuPG. Say you are on a large keyring and on a train, then that keyring is first passed through your enterprise malware protection for scanning or something like that. Then it works again until some metric, hash or something else changes.

My recommendation would at this point be to use procmon with a file filter for just "If path contains gnupg then include" I mean maybe go only for the locking dirs but this way you will not only see what the GnuPG processes are doing but what everyone on the system is doing to the locks. So you will see when my old friends, third party security software might interfere.
For example: You will see on a default Windows which files are checked through telemetry. And here in this example you see directly that the Microsoft Malware Protection Engine is accessing the agents socket.

Spent some time discovering and unfortunately it's Windows's bug in loopback interface.
I wrote a test demo (blocking mode) to exchange data and watched their packets, found that network stack would drop packets when congestion control algorithm is set to BBR2. It seems the second data exchange was broken.

In T5993#201111, @werner wrote:

For example Poppler uses GnuPG comment packets to lower its own attack surface by leaving all OpenPGP handling to gpg. The patch (or at least the version we noticed in Fedora and Debian) entirely breaks this use.

(The commits had a wrong bug it in their message)

It might be useful to have samples of compressed keys:

No, we can't do much about this. It has always been easy to create compression bombs and the more relevant thing here is compressed signed or encrypted data. Or just compressed mails. The patch by @DemiMarie is way to complicated for what it wants to achieve and actually breaks existing use cases. For example Poppler uses GnuPG comment packets to lower its own attack surface by leaving all OpenPGP handling to gpg. The patch (or at least the version we noticed in Fedora and Debian) entirely breaks this use.

Using the primary key for ssh was not intended and thus not tested. I have not yet found the time too look closer at your report. Just one remark:

In T7620#200845, @Saturneric wrote:

I think it would be much better if GnuPG automatically performed a key listing immediately after key generation when a smartcard is involved. This would allow GnuPG to detect the presence of the subkey on the card right away, rather than leaving it marked as a stub until the user manually lists keys.

I see that you generated the secret encryption subkey with backup. This means that the secret subkey is generated on your computer, then copied to the card, and then deleted from your computer. The deletion is the reason why the subkey is marked as stub. Only after listing the keys on the card gpg notices that the secret key is actually on the card.