In Kleopatra we explicitly trigger a re-reading of the smart card after each operation involving a smart card to ensure that Kleopatra doesn't show wrong information. There's so much that can go wrong with physical smart cards that this is the only way to make sure you don't tell the user lies. I think gpg --edit-card also re-reads the smart card after each operation.

There is no bug in the contexts and there's nothing to document anywhere. If anything then it's a bug in gpg's generate command or a more general issue (in gpg-agent) with keeping track of the storage location of private keys as I have already explained in T7620#200613. I'm removing the gpgme tag because there's nothing wrong in gpgme and there's nothing we can do in gpgme. It needs to be addressed in gnupg.

In practice, calling gpgme_get_key() will often pick up most changes because GPGME asks the underlying GPG agent daemon, which may re-read the keyring. That gives the impression that a long-lived context automatically reflects live updates. However, as aheinecke noted, some updates can still go unnoticed in a single gpgme_ctx_t, so it isn’t a strictly frozen snapshot nor a perfectly live view—behaviors are mixed.

In T7620#201528, @aheinecke wrote:

Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

The problem was: In scdaemon, PKSIGN with OPENPGP.3 didn't work well for Ed25519 (done by do_auth function in app-openpgp.c), when --hash=sha512 (not SHA1).

I located the bug in scdaemon.

Pushed the change: rG16ee68259d1d: gpg,regexp: Use -DREGEXP_PREFIX=gnupg_.

I do not think that this is the only place where such an issue occurs. Maybe we should make the documentation clearer about context key reuse. But the context is specifically designed to cache information about a key, so as to avoid memory overhead. I learned early on that its best for each new operation to use a new context. A context is basically an instance of gpg or gpgsm. So you start one process, ask it for a keylist, keep the process running, start another process, modify the key database, and then ask the first process again about his worldview. Either the first process is a bit confused because it has read data and then that data changed (what happens here) or it has no idea about the change since it was efficient and only read the database once. But here in this example you should be able to reproduce this also by making any other modifications to the key, adding other subkeys, userids etc. That GPGME even notices the secret key is more of a side effect of how the programming works because the GPGME gpg process will ask the gpg-agent (so a third process).

The more I think of this, the more likely this appears to me as the source for all that random startup weirdness of GnuPG. Say you are on a large keyring and on a train, then that keyring is first passed through your enterprise malware protection for scanning or something like that. Then it works again until some metric, hash or something else changes.

My recommendation would at this point be to use procmon with a file filter for just "If path contains gnupg then include" I mean maybe go only for the locking dirs but this way you will not only see what the GnuPG processes are doing but what everyone on the system is doing to the locks. So you will see when my old friends, third party security software might interfere.
For example: You will see on a default Windows which files are checked through telemetry. And here in this example you see directly that the Microsoft Malware Protection Engine is accessing the agents socket.

Spent some time discovering and unfortunately it's Windows's bug in loopback interface.
I wrote a test demo (blocking mode) to exchange data and watched their packets, found that network stack would drop packets when congestion control algorithm is set to BBR2. It seems the second data exchange was broken.

In T5993#201111, @werner wrote:

For example Poppler uses GnuPG comment packets to lower its own attack surface by leaving all OpenPGP handling to gpg. The patch (or at least the version we noticed in Fedora and Debian) entirely breaks this use.

(The commits had a wrong bug it in their message)

It might be useful to have samples of compressed keys:

No, we can't do much about this. It has always been easy to create compression bombs and the more relevant thing here is compressed signed or encrypted data. Or just compressed mails. The patch by @DemiMarie is way to complicated for what it wants to achieve and actually breaks existing use cases. For example Poppler uses GnuPG comment packets to lower its own attack surface by leaving all OpenPGP handling to gpg. The patch (or at least the version we noticed in Fedora and Debian) entirely breaks this use.

Using the primary key for ssh was not intended and thus not tested. I have not yet found the time too look closer at your report. Just one remark:

In T7620#200845, @Saturneric wrote:

I think it would be much better if GnuPG automatically performed a key listing immediately after key generation when a smartcard is involved. This would allow GnuPG to detect the presence of the subkey on the card right away, rather than leaving it marked as a stub until the user manually lists keys.

I see that you generated the secret encryption subkey with backup. This means that the secret subkey is generated on your computer, then copied to the card, and then deleted from your computer. The deletion is the reason why the subkey is marked as stub. Only after listing the keys on the card gpg notices that the secret key is actually on the card.

btw, my clue was that in that last --check-sigs, if i used --debug-all i got this:

This affects certification-only primary keys when doing web-of-trust calculations.

Hi Werner, I submitted a patch right after this bug report using AC_CHECK_DECLS([_sys_siglist]) [1].

To avoid further noise on this ticket, i've done as requested and posted to gnupg-devel : https://lists.gnupg.org/pipermail/gnupg-devel/2025-May/035875.html

The first call of get_key receives the following key listing from gpg:

2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: sec:-:256:19:C4A24EB0B5F2E025:1746474606:::u:::s
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: cESCA:::D2760001240100000006180489130000::brainp
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: oolP256r1:23::0:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: fpr:::::::::DEC0948C398A6E7B50746EC6C4A24EB0B5F2
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: E025:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: grp:::::::::06BDACFBDEDBC5783A75AE5E7251FA3369C4
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: 0FF4:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: uid:-::::1746474606::2222D8E2F373B9BDEE0DEA2A20A
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: 9402214E9F984::Eric <eric@bktus.com>::::::::::0:
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: <LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: ssb:-:256:19:EAFC5EA29B758B22:1746474606::::::a:
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: ::D2760001240100000006180489130000::brainpoolP25
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: 6r1:23:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: fpr:::::::::1AD596DDEC9B8CF3C1AC6C41EAFC5EA29B75
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: 8B22:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: grp:::::::::52F0797C0B0439BBD718E2534D46656A6C45
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: 6A78:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: ssb:-:256:18:A874804DB497B91C:1746474606::::::e:
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: ::#::brainpoolP256r1:23:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: fpr:::::::::33B273C7BD46E4EB63DD6874A874804DB497
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: B91C:<LF>
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: grp:::::::::34A1F8D9B2AA0CF07C2E042D70E10F9D4EBE
2025-05-05 21:50:23 gpgme[57059]     _gpgme_io_read: check: E734:<LF>

Note the line

ssb:-:256:18:A874804DB497B91C:1746474606::::::e:::#::brainpoolP256r1:23:<LF>

where the # marks the subkey as stub.

Right now we have

Interesting, that sounds like a portable method. I am not very familiar with GPG internals, but to me that sounds like quite a bit of work. Unless there is another benefit to doing so, I don't think it is worth it just to print signal names.

I have now identified the exact conditions and a reproducible path for the issue I previously reported. I will also attach the relevant gpgme.log.