Page MenuHome GnuPG

Kleopatra: encryption via "Encrypt folder" has unicode issue
Closed, ResolvedPublic


In Kleopatra: File > Sign/Encrypt folder, choose a folder with a > 8bit unicode character:

Looks good this far

Then click Sign/Encrypt. Without a pinentry window popup you get immediatly:

[1448] org.kde.pim.kleopatra: Kleo::Crypto::SignEncryptTask(0x12382148) slotResult job: QGpgME::QGpgMESignEncryptArchiveJob(0x9811188) signing result: "GpgME::SigningResult(\n error: GpgME::Error(1 (Allgemeiner Fehler))\n createdSignatures:\n invalidSigningKeys:\n)" encryption result: "GpgME::EncryptionResult(\n error: GpgME::Error(1 (Allgemeiner Fehler))\n invalid recipients:\n)"


gpg: enabled debug flags: memstat
gpg: enabled compatibility flags:
gpg: der Unterschlüssel FFA2FCCB2EC589F8 wird anstelle des Hauptschlüssels 458612006D8E6F0D verwendet
gpg: No trust check due to '--trust-model always' option
gpg: 'C:/Users/g10code.WIN-TEST3/Documents/Ordner ??.tar.gpg' kann nicht erzeugt werden: Input/output error
gpg: [stdin]: sign+encrypt failed: Input/output error
gpg: keydb: handles=2 locks=0 parse=2 get=2
gpg:        build=0 update=0 insert=0 delete=0
gpg:        reset=0 found=2 not=0 cache=0 not=0
gpg: kid_not_found_cache: count=0 peak=0 flushes=0
gpg: sig_cache: total=4 cached=4 good=4 bad=0
gpg: random usage: poolsize=600 mixed=1 polls=0/4 added=81/1044
              outmix=0 getlvl1=0/0 getlvl2=0/0
gpg: rndjent stat: collector=0x00000000 calls=0 bytes=0
gpg: secmem usage: 1344/32768 bytes in 2 blocks

This is a regression, it works in VSD 3.1.26 and Gpg4win-4.1.0!


VS-Desktop- and Gpg4win-4.2.0

Event Timeline

For clarification: encrypting a file via Kleopatra (not GpgEX) with unicode characters in name works, only the folders don't

ebo changed Version from VS-Desktop- to VS-Desktop- and Gpg4win-4.2.0.Sep 20 2023, 5:44 PM
aheinecke triaged this task as High priority.
aheinecke added a subscriber: aheinecke.

Ingo can you investigate? This is very important because this is a regression caused by the change to pass filenames directly. I wonder if we pass this as UTF-8 and gpgtar needs a fix or if we pass it somehow else.

This is a release blocker for me.

ikloecker moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Sep 21 2023, 11:16 AM

Encrypting folders with umlauts in the folder name and decrypting archives with umlauts in the file name now works for me on Windows. Encrypting folders and decrypting archives with kanji in the names still fails.

Encrypting the folder ordner-日本国-kanji fails with

gpg: 'C:/Users/g10code/Desktop/testdata/ordner-???-kanji.tar.gpg' kann nicht erzeugt werden: Input/output error.
gpg: [stdin]: sign+encrypt failed: Input/output error.

Decrypting the archive ordner-日本国-kanji.tar.gpg (renamed archive with ASCII-only folder) fails with

gpgtar: error creating directory 'C:/Users/g10code/AppData/Local/Temp/kleopatra-FKhkiO/ordner-ascii-???-kanji/ordner-ascii': Input/output error

Relevant gpgme logs for encryption:

2023-09-22 10:13:14 gpgme[2056.634] gpgme_data_set_file_name: enter: dh=0x0794f448 file_name=C:/Users/g10code/Desktop/testdata/ordner-日本囜-kanji.tar.gpg
2023-09-22 10:13:14 gpgme[2056.634]     gpgme_data_get_file_name: call: dh=0x0794f448 dh->file_name=C:/Users/g10code/Desktop/testdata/ordner-日本囜-kanji.tar.gpg
2023-09-22 10:13:14 gpgme[2056.634]     _gpgme_io_spawn: check: path=0x078eb510 argv[23] = C:/Users/g10code/Desktop/testdata/ordner-日本囜-kanji.tar.gpg
2023-09-22 10:13:14 gpgme[2056.634]       _gpgme_io_write: check: ordner-.........-kanji

As far as I can tell, the file name of the archive is added correctly encoded as UTF-8 to the command line arguments for calling gpgtar. Therefore, I think there is another problem in gpgme (_gpgme_io_spawn) or gpgme-w32spawn.exe or gpgtar. @werner ?

ikloecker added a subscriber: ikloecker.

Somebody else needs to take over.

ikloecker mentioned this in Unknown Object (Event).Sep 25 2023, 9:28 AM

Yes I can see that gpgtar correcly lists the directory with procmon, accesses the files but fails to create the output file indeed.

This is the command line gpgtar is called with according to procmon (note that it already includes the question marks):

"C:\\\\Program Files (x86)\\\\GnuPG VS-Desktop\\\\GnuPG\\\\bin\\\\gpgtar.exe" "--batch" "--status-fd" "4" "--gpg-args" "--no-tty" "--gpg-args" "--charset=utf8" "--gpg-args" "--enable-progress-filter" "--gpg-args" "--exit-on-status-write-error" "--gpg-args" "--ttyname=/dev/tty" "--encrypt" "--sign" "--gpg-args" "--always-trust" "-r" "4616692EEEDCB736B380FCA2D78D8BB4CA553DA3" "-u" "D78D8BB4CA553DA3" "--output" "C:/Users/Andre Heinecke/Desktop/Test??Test.tar.gpg" "--directory" "C:/Users/Andre Heinecke/Desktop" "--files-from" "-" "--null" "--utf8-strings"

We can't really compare this with GnuPG since there we write into a QFile created by Kleopatra which we also did with gpgtar in the past.

ebo mentioned this in Unknown Object (Event).Oct 2 2023, 9:50 AM

This one is one me. I think the issue is gpgme-w32 spawn.

So I have analyzed the problem and I think I understand it now, but I don't really have a solution yet as I think a new option or change in gpgtar might be needed. I think the easiest would be that if --utf8-strings is provided that also the --output parameter is assumed to be UTF8 encoded? And not just the files from --files-from?

So the problem is since GPGME is a general purpose library we cannot really enforce where users install it to. (The backdrop for this is ) So GPGME can be installed into some users home directory with cryillic characters for example. GPGME needs to then call gpgme-w32-spawn next to it. For this it then uses CreateProcessW and passes wchar arguments. But since gpgme-w32-spawn is not compiled with -municode it only has a 8bit main. So windows converts the utf16 args to local 8 bit, which fails for unrepresentable characters.

So the solution then is to extend gpgme-w32-spwan with a wmain and compile it with -municode so that Unicode arguments can be passed. Now the output of procmon looks like this:

"C:\\\\GnuPG\\\\bin\\\\gpgtar.exe" "--batch" "--status-fd" "4" "--gpg-args" "--no-tty" "--gpg-args" "--charset=utf8" "--gpg-args" "--enable-progress-filter" "--gpg-args" "--exit-on-status-write-error" "--gpg-args" "--ttyname=/dev/tty" "--encrypt" "--sign" "--gpg-args" "--armor" "--gpg-args" "--always-trust" "-r" "4616692EEEDCB736B380FCA2D78D8BB4CA553DA3" "-u" "D78D8BB4CA553DA3" "--output" "C:/Users/Andre Heinecke/Desktop/Test🍔Test.tar.asc" "--directory" "C:/Users/Andre Heinecke/Desktop" "--files-from" "-" "--null" "--utf8-strings"

This looks correct, since the UTF-8 Character 🍔 is f09f 8d94 0a

Please note that procmon would either show UTF16 or cp-1252

And in cp1252
f0 is: ð
9f is: Ÿ
and so on.

But the problem then is that gpgtar, of course cannot know that in this case it actually has gotten the output filename as utf-8 string and writes this file then out according to cp1252.

So I think we need to either add an option to gpgtar to pass utf8 filenames or to use a wmain altogether, *juk*

Attached is the patch to make gpgme-w32-spawn unicode aware.

According to werner the gnupg tools use GetCommandLineW even when they are not build with -municode.
So a solution could be to build gpgme-w32-spawn with -municode and start the child process with CreateProcessW, this would also solve the problem that GnuPG could itself be installed into Paths which are not representable in the local 8 bit encoding.

aheinecke changed the task status from Open to Testing.Oct 25 2023, 5:46 PM

This works now. Tested both decrypt and encrypt. Sadly just one commit after GPGME 1.23.0 but this was a miscommunication because I was a bit unavailable :( But we can patch this into our installer.

So workflow is now testing but WIP until a testable version is provided, right?

aheinecke mentioned this in Unknown Object (Event).Oct 30 2023, 8:31 AM
ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Oct 31 2023, 2:07 PM
ebo removed a project: gnupg22.
ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.

works with VS-Desktop-