Found this one in Gpg4win-4.2.0-beta384:

Found in GnuPG-VS-Desktop-3.2.0-beta178, broken ü and no linebreaks:

Setting this to high after consultation with @werner, these issues should be fixed in next release.

In T5960#175302, @ebo wrote:

Found in GnuPG-VS-Desktop-3.2.0-beta178, broken ü and no linebreaks:

Setting this to high after consultation with @werner, these issues should be fixed in next release.

This is export to server. This one seems to come from a direct call to gpg instead of using gpgme. Very bad.

The debug output of libkleo is missing. Please add org.kde.pim.libkleo.debug=true to your qtlogging.ini.

With regard to the ü-error in the error message on keytocard somebody else has to check what's going wrong because I'm convinced that we are doing everything right.

In gpgme (src/version.c) we call

gettext_use_utf8 (1);

which is supposed to force libgpg-error's gettext to return UTF-8 encoded messages.

And in libkleo Formatting::errorAsString which is used for the error message on keytocard we do

#if defined(Q_OS_WIN) && GPGMEPP_ERROR_ASSTRING_RETURNS_UTF8_ON_WINDOWS
    return QString::fromUtf8(error.asString());
#else
    return QString::fromLocal8Bit(error.asString());
#endif

I'm blind to seeing where this goes wrong.

Edit: The above changes have landed in the code in April 2023. It should be double-checked that Gpg4win-4.2.0-beta384 didn't use older code. Best check again with the latest build.

The previous two changes will hopefully fix the error output of export to server. I'm not entirely sure about the missing line breaks.

Hopefully, we will soon have a Windows CI to catch such errors.

In T5960#175318, @ikloecker wrote:

Also, please retest T5960#172860 with the latest build.

The ü there is still broken in Gpg4win-4.2.1-beta31 build today with up to date libkleo (but 3 days old gpgme)

[6064] org.kde.pim.kleopatra: KeyToCardCommand::doStart()
[6064] org.kde.pim.kleopatra: KeyToCardCommand::Private::start()
[6064] org.kde.pim.kleopatra: ReaderStatus::getCard() - Found card with serial number "D2760001240100000006154932980000" and app "openpgp"
[6064] org.kde.pim.kleopatra: KeyToCardCommand::Private::startKeyToOpenPGPCard()
[6064] org.kde.pim.kleopatra: ReaderStatus::getCard() - Found card with serial number "D2760001240100000006154932980000" and app "openpgp"
[6064] org.kde.pim.kleopatra: ReaderStatusThread[2nd]: new iteration command= "KEYTOCARD --force 8B8713E028E9DC05A4FEA1B70A21B293C2081819 D2760001240100000006154932980000 OPENPGP.2 20230308T111545"  ; nullSlot= false
[6064] org.kde.pim.libkleo: sendCommand "SCD SWITCHCARD D2760001240100000006154932980000"
[6064] org.kde.pim.libkleo: sendStatusLinesCommand "SCD SWITCHCARD D2760001240100000006154932980000" : got ( status( "SERIALNO" ) = "D2760001240100000006154932980000" 
[6064]  )
[6064] org.kde.pim.libkleo: sendCommand "SCD SWITCHAPP openpgp"
[6064] org.kde.pim.libkleo: sendStatusLinesCommand "SCD SWITCHAPP openpgp" : got ( status( "SERIALNO" ) = "D2760001240100000006154932980000 openpgp piv" 
[6064]  )
[6064] org.kde.pim.libkleo: sendCommand "KEYTOCARD --force 8B8713E028E9DC05A4FEA1B70A21B293C2081819 D2760001240100000006154932980000 OPENPGP.2 20230308T111545"
[6064] org.kde.pim.libkleo: sendCommand "KEYTOCARD --force 8B8713E028E9DC05A4FEA1B70A21B293C2081819 D2760001240100000006154932980000 OPENPGP.2 20230308T111545" failed: Ung?ltiger Wert (code: 55, source: SCD)

Then gettext_use_utf8 doesn't seem to be working as intended.

In T5960#175407, @ebo wrote:

The result of your changes is a bit unexpected for me, maybe something is missing in the testversion Gpg4win-4.2.1-beta31?

[6064] org.kde.pim.libkleo: gpgConfGetConsoleOutputCodePage returns 65001
[6064] org.kde.pim.libkleo: stringFromGpgOutput trying to decode "" using codepage 65001

Looks like the output of the command is empty. Maybe a timing problem?

[6064] kf.i18n.kuit: "Markup error in message {<__kuit_internal_top__><para>Beim Export der OpenPGP-Zertifikate ist ein Fehler ...}: Unerwartet ''.. Last tag parsed: message. Complete message follows:\n<__kuit_internal_top__><para>Beim Export der OpenPGP-Zertifikate ist ein Fehler aufgetreten.</para> <para>Die Ausgabe von <command>C:\\Program Files (x86)\\GnuPG\\bin\\gpg.exe</command> lautet: <message>\u0000</message></para></__kuit_internal_top__>"

Looks like we decode the string including the terminating null character.

This looks to me like the typical thing where you also do not get the "Diagnostics" button viewed. Because log-file is set in the various .conf files and the stderr does not get back to GPGME / Kleopatra so we have nothing to show there because the actual error ended up in a log file.

ok, with the new testversion VS-Desktop-3.2.0.0-beta214 and without Debug redirection (mea culpa), the info window looks like this:

[3376] org.kde.pim.libkleo: gpgConfGetConsoleOutputCodePage starting "C:\\Program Files (x86)\\GnuPG VS-Desktop\\bin\\..\\GnuPG\\bin\\gpgconf.exe" --show-codepages
[3376] org.kde.pim.libkleo: gpgConfGetConsoleOutputCodePage ConsoleOutputCP is 0 - use ACP
[3376] org.kde.pim.libkleo: gpgConfGetConsoleOutputCodePage returns 0

I tried to reproduce this with a locally built gpg4win-4.2.1-beta56. Using localhost as keyserver I got (after a looooooong timeout)

with debug output

[6472] org.kde.pim.libkleo: gpgConfGetConsoleOutputCodePage starting "C:\\Program Files (x86)\\GnuPG\\bin\\gpgconf.exe" --show-codepages
[6472] org.kde.pim.libkleo: gpgConfGetConsoleOutputCodePage returns 65001
[...]
[6472] org.kde.pim.kleopatra: "C:\\Program Files (x86)\\GnuPG\\bin\\gpg.exe --send-keys 40C8670A86AC2B2C35AC6E1B25579E2EBB089E1B gestartet ..."
[...]
[6472] org.kde.pim.libkleo: stringFromGpgOutput trying to decode "gpg: sende Schl\xC3\xBCssel 25579E2EBB089E1B auf hkps://localhost" using codepage 65001
[6472] org.kde.pim.kleopatra: "gpg: sende Schlüssel 25579E2EBB089E1B auf hkps://localhost"
[6472] org.kde.pim.libkleo: stringFromGpgOutput trying to decode "gpg: Senden an Schl\xC3\xBCsselserver fehlgeschlagen: Unknown error\r\ngpg: Senden an Schl\xC3\xBCsselserver fehlgeschlagen: Unknown error" using codepage 65001
[6472] org.kde.pim.kleopatra: "gpg: Senden an Schlüsselserver fehlgeschlagen: Unknown error\ngpg: Senden an Schlüsselserver fehlgeschlagen: Unknown error"
[6472] org.kde.pim.libkleo: stringFromGpgOutput trying to decode "gpg: sende Schl\xC3\xBCssel 25579E2EBB089E1B auf hkps://localhost\r\ngpg: Senden an Schl\xC3\xBCsselserver fehlgeschlagen: Unknown error\r\ngpg: Senden an Schl\xC3\xBCsselserver fehlgeschlagen: Unknown error\r\n" using codepage 65001
[6472] org.kde.pim.libkleo: stringFromGpgOutput trying to decode "gpg: sende Schl\xC3\xBCssel 25579E2EBB089E1B auf hkps://localhost\r\ngpg: Senden an Schl\xC3\xBCsselserver fehlgeschlagen: Unknown error\r\ngpg: Senden an Schl\xC3\xBCsselserver fehlgeschlagen: Unknown error\r\n" using codepage 65001

The remaining problem with output that's read directly from a GnuPG process seems to be that sometimes gpgconf --show-codepages prints 0 as ConsoleOutputCodePage or even as ACP (as in T5960#175580).

If a valid ConsoleOutputCodePage is printed by gpgconf as in my previous comment, then the decoding works as intended.

I have no idea why gpgconf sometimes fails to print valid code pages. Maybe it depends on the way Kleopatra is started. I started it by double clicking the application icon on the desktop and by typing kleopatra in the Search field.

With regard to invalid characters in error messages as in T5960#172860:
We are using the identical function everywhere to convert the error codes to text. If an invalid character is displayed if copying a key to a smart card failed, then invalid characters should also be displayed for example if one tries to decrypt something in the notepad without having the necessary secret key. I get "Kein geheimer Schlüssel" in this case, i.e. no invalid character.

In the VSD Testversion:

C:\Users\g10code.WIN-TEST3>gpgconf --show-codepages
Ungültige Option "--show-codepages"

So this seems to be an issue of the gpg version in VSD

gpgconf --show-codepages ist just a debugging aid. We use the code pages only for output to the console. The problem we see here is about log messages and they are always send as utf8 to stderr or the pipe used instead - without any translation.

From our always used init_common_subsystems:

#ifdef HAVE_W32_SYSTEM
  /* We want gettext to always output UTF-8 and we put the console in
   * utf-8 mode.  */
  gettext_use_utf8 (1);
  if (!SetConsoleCP (CP_UTF8) || !SetConsoleOutputCP (CP_UTF8))

See also T4365 and rGb912f07cdf (gnupg-2.2)

With the latest change the output read directly from gpg should now be displayed correctly.

Error messages were already displayed correctly for me on Windows (T5960#175623).

This looks nice! One down...

But "Zeitüberschreitung" is still broken. (See pictures above.)
For retest of "ungültiger Wert" I would need a newer gpg4win Testversion.

with VS-Desktop-3.1.90.258-Beta the ü is still broken for "Zeitüberschreitung" in the error window which occurs in the context of keytocard:

[5780] org.kde.pim.libkleo: sendCommand "KEYTOCARD --force 1A63F7F35AB5F31A7BE34555DE616C0A2E48F32E D276000124010304000500009D400000 OPENPGP.1 20230308T111343" failed: Zeit?berschreitung (code: 62, source: Pinentry)

In the context of e.g. signing there is another kind of window, where the "ü" is displayed correctly.

I tried to reproduce this but I didn't succeed. I used "Create New OpenPGP Cert" for my attempts.

First try with unpatched error dialog. Note the slashes surrouding the error text.

Next try with added translation context "@info" which makes xi18nc output HTML. Note: Now the error text is printed italic. But the error message is not translated (because of the added translation context).

Next try with patched translations adding the message with context. Done to rule out (encoding of) translation playing a role in the issue.

Next try with the exact error message the encoding issue occurred with (but still with OpenPGP certificate creation).

Final try with the exact same code for the message box.

I have no idea why it should make a difference whether the error message is shown after OpenPGP certificate creation or after copying a key to a smart card. Everything should run in the main thread. I'll add some more debug output for further investigation.

I have no idea why it should make a difference whether the error message is shown after OpenPGP certificate creation or after copying a key to a smart card. Everything should run in the main thread. I'll add some more debug output for further investigation.

The one goes through GPGME and the other is talking directly to the agent so there is a big difference.

Sure. But in both cases they get back an error code and then, back in safe Kleopatra territory, they convert this error code into a string. I don't see how this error code to string conversion should have anything to do with the way the commands triggered the operation. Both conversions should happen in the main thread, so that the state of libgpg-error should be identical. The logging will show if in the still failing case libgpg-error is not switched to UTF-8 for localized texts.

Some logs after entering a wrong reset code (PUK) for an OpenPGP smart card.

404.228467	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: sendCommand "SCD PASSWD --reset OPENPGP.2" failed: "Falscher R?ckstellcode" (code: 322, source: SCD)
404.230320	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: errorAsString gettext_use_utf8(-1) returns 1
404.230596	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: errorAsString error: Falscher R?ckstellcode
404.231068	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: errorAsString error (percent-encoded): "Falscher%20R%FCckstellcode"
404.231182	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.kleopatra: ChangePinCommand::slotResult(): "Falscher R?ckstellcode" ( 322 )
404.231315	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: errorAsString gettext_use_utf8(-1) returns 1
404.231428	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: errorAsString error: Falscher R?ckstellcode
404.231850	2024/05/27 15:56:17.987	8056	kleopatra.exe	org.kde.pim.libkleo: errorAsString error (percent-encoded): "Falscher%20R%FCckstellcode"

I think I have identified the root cause of the remaining issue with wrongly encoded error texts. The good thing is that the problem shouldn't occur if one does not enable debug logging for libkleo/kleopatra.

Analysis (Windows only):

• GpgME's Error::asString is called from the ReaderStatus thread to output the log message [...] sendCommand [...] failed: [...] shown above.
• On the first call Error::asString calls gpgme_strerror_r and stores the result in a member variable.
• gpgme_strerror_r indirectly calls get_string (libgpg-error/src/w32-gettext.c).
• get_string checks tls->gt_use_utf8 to determine whether the translation needs to be converted from UTF-8 to native encoding.
• tls stands for "thread-local storage".
• For each thread a tls instance is created (DllMain in libgpg-error/src/init.c). tls->gt_use_utf8 is initialized as 0.
• On initialization, gpgme calls gettext_use_utf8(1) which sets tls->gt_use_utf8 to 1. But only for the main thread!

Conclusion: If Error::asString is called from a secondary thread then it caches a wrongly encoded error message. This cached error message is then returned on subsequent calls of Error::asString from the main thread.

I will submit a bug for libgpg-error because I think for secondary threads tls->gt_use_utf8 should be initialized to the value of the main thread. And I will add an alternative to Error::asString which doesn't cache the result of gpgme_strerror_r. Unfortunately, Error::asString is unfixable because it returns a const char * which wouldn't be thread-safe if the pointer didn't point to the cached member variable. It's ironic that a fix for making Error::asString more thread-safe introduced a bug if it's called from different threads.

Update: The bug for libgpg-error is T7185: Calling gettext_use_utf8(1) for the main thread should enable UTF-8 for all later created secondary threads.

Mark for backport to VSD 3.3

Now even the error descriptions that are logged by background threads should have readable umlauts (see T7188).

with gpg4win-Beta-50: "Rückstellcode" is shown correctly with an ü