Gpgtar fails when files have non ASCII characters
Closed, ResolvedPublic

Description

Hello,

If I use gpgex to encrypt/decrypt a file with a german umlaut, there comes a GPA
Error "Invalied argument". After renaming the file the encrpyt/decrypt completes
without error. I did install GPG4win 2.2.1 on Windows 7 Professional 32 bit.
This error comes also if an umlaut is in the directory strukture included.

Here you can copy some ä ö ü ß

Details

Version
gpg4win 2.2.3
boehmtho set Version to gpg4win 2.2.1.Mar 21 2014, 4:30 PM
boehmtho added projects: gpgex, Bug Report.
boehmtho added a subscriber: boehmtho.

I can reproduce this on my test system, if I just create a folder with the name
"földer" Kleopatra / gpg2 /gpgtar don't show an error they just fail silently
and create an empty archive. I guess there is some utf8 to system locale
conversion missing.

As kleopatra offers me the option to "Delete unencrypted files" after encryption
I am marking this as critical as it can lead to data loss.

At least there should be an error so that the removal after the encryption is
not done!

aheinecke raised the priority of this task from High to Unbreak Now!.
aheinecke changed Version from gpg4win 2.2.1 to gpg4win 2.2.2.
aheinecke claimed this task.
aheinecke added subscribers: emanuel, werner.

accidentally removed the original reporter from the nosy. Sorry (readded)

bernhard renamed this task from gpgex does not work with umlaut to file encryption: gpgex and gpgtar does not work with umlaut.Oct 23 2014, 11:45 AM

gpgtar correctly returns an exit code != 0 in case an error happens. This exit
code is eaten by kleopatra so the problem that success is reported on error is
on kleos side.

But gpgtar has the encoding problems. From the kleopatra side it looks ok file
names are handed over to gpgtar in local 8 bit encoding and not in utf-8. And
even directly on the command line it fails.

C:\Users\aheinecke\Desktop>"c:\Program Files\GNU\GnuPG\gpgtar.exe" --skip-crypto
--verbose -o c:\Users\aheinecke\Desktop\test.tar -e fäil.txt
gpgtar: error stat-ing `fΣil.txt': The system cannot find the file specified.

The problem is that internally gpgtar treats argument file names as UTF-8 and
even converts the return value from syscalls like findfirstfile to UTF-8 before
opening them. The open uses the utf8 encoded filename and fails as the file
system usually does not use utf8 file names on Windows.

A workaround would be to convert to ACP (CP_ACP (afaik ACP is correct for
filenames)) instead of converting to UTF-8. But this will not work for 💩
but as gpgex and kleopatra already fail earlier on "Non 8 bit representable
characters" this should be acceptable for now.

So two fixes need to be done:

  • Use system 8 bit encoding for open in gpgtar. -> Should fix most problems
  • Handle the return value of gpgtar correctly and escalate it to the user in

kleopatra.

  -> Should make the rest of the problems "uncritical"

I hope to get this done tomorrow evening.

Thank you for working on this problem. I also use GPA 0.9.4 for Windows as gpg
frontend. Would be great if the fix will make the file encryption working with
umlaut also in GPA and kleopatra.

Thank you & Regards
boehmtho

aheinecke reassigned this task from aheinecke to werner.Oct 24 2014, 6:18 PM

Werner: Please see attached Patch.

I've tested this with kleopatra and it works now to encrypt a folder which has
special characters in the local code page in filenames. And tested it on the
command line.

This should improve the status quo a lot as full utf16 file names are rarer and
gpgtar / gpg4win is not the only tool which has problems with utf16 filenames ;-)

I did not go for proper unicode support because this would mean:

  1. Convert all command line arguments from native to utf8. (Or even to expect

them to be utf-16 *brr*)

  1. Expect files-from / stdio file names to be utf8 encoded (and update the callers)
  2. Use wide char file io functions. Which would add lots of #ifdefs.

But we would have to discuss if we should do 2. also on non-Windows systems.
Currently gpgtar expects local 8 bit encoding there. So tools using gpgtar would
have to convert their arguments differently for Windows.

The error is now handled by Kleopatra:

http://commits.kde.org/kdepim/2a58b4cb452cdb132553c2381ce810bbc2606e55

I'm a bit scared of regressions, though as the Input handling is so
"generalized" in kleo that I don't know if I now broke cases where input errors
are expected :-/

Attached is a screenshot how it looks now with a broken gpgtar version. And
files are no longer deleted as the operation is no longer thought to be successful.

I did install gpg4win 2.2.3 which does include the fix for Umlaut. I prefer to
use GPA as frontend for encryption and decryption. For decryption, I did link
.gpg to be always opened with GPA. Unfortunatly if the File or Path to the .gpg
has an Umlaut included the GPA GUI crashes. Only if I use the open button from
GPA the file can be added to the GPA frontend and decrypted. For non-umlaut
files the link to open .gpg works fine. Do you also work on GPA bugs or do I
have to report this under the GPA category?
Thank you & regards,

aheinecke changed Version from gpg4win 2.2.2 to gpg4win 2.2.3.Nov 28 2014, 3:12 PM
aheinecke added a project: gpa.
aheinecke removed projects: gpgtar, gpgex.
aheinecke claimed this task.

I've changed the category to gpa, adjusted the topic and version to 2.2.3
As you've already described the problem together with GPA here I think this is
better then opening a new bug.

I'll also no longer call this critical as the original data loss problem
(Encrypting files where one has an umlaut -> kleo thinks its a success and
deletes the original) Should be resolved.

The fix in GPA should be fairly easy. Some conversion from native to utf-8 on
input and utf-8 to native on output. So I'm taking this issue.

Werner: Could you please take a look at the patch for gpgtar. I will probably
propose something quite similar for GPA. Not real unicode support but at least
for 8 bit filenames.

aheinecke renamed this task from file encryption: gpgex and gpgtar does not work with umlaut to file encryption: gpa does not work with umlaut.Nov 28 2014, 3:12 PM
bernhard lowered the priority of this task from Unbreak Now! to High.Jan 8 2015, 11:58 AM
bernhard reassigned this task from aheinecke to werner.

It probably would have been better to create two issues:
a) Dataloss with Kleo in 2.2.2 (fixed now)
b) crash with gpa

I've opened T2185 for the GPA Problem so i can change the topic here and we
can cleanly close this issue when the gpgtar fix is applied upstream.

We might also want to create a new fix for UTF-16 support in gpgtar once this is
closed. But the attached patch would improve the current situation already a lot.

aheinecke renamed this task from file encryption: gpa does not work with umlaut to Gpgtar fails when files have non ASCII characters.Dec 11 2015, 11:53 AM

Btw. this was patched in Gpg4win for over a year now. So I expect we would have
heard if this caused regressions.

aheinecke added a project: gnupg.

Hello Andre,

GPA is still crashing during open a file (file > open) with umlaut.
I am using GPA 0.9.9 from GPG4win 2.3.0

Please try to reproduce. Here you can get some umlauts ö ä ü ß.

Also if you have associated .gpg extension to be opened with GPA in
windows. The GUI does not crash instead the file is not added to the
open > list and cannot be decrypted - compared to a file without umlaut's.
Please find screenshots attached.
The solution is to remove the umlaut. However, this is not a great usage
acceptance especially for German gpg4win users.

Regards,
Thorsten Böhm

Computergenerierter Alternativtext: Datei öffnen Suchen Zuletzt
verwendet Name listgpg.txt md5.7z.txt . nexsen É60.Icg.gpg Letzte
Änderung* Größe 217 26062014 2,5 k8 46:2 k-B 14082015 26.06.2314

Computergenerierter Alternativtext: gpa.exe gpa.exe funktioniert nicht
mehr Ein Problem hat die richtige Ausführung dieses Programms
verhindert. Schließen Sie das Programm. Programm schließen

Andre Heinecke via BTS schrieb:

Andre Heinecke <aheinecke@intevation.de> added the comment:

Btw. this was patched in Gpg4win for over a year now. So I expect we would have
heard if this caused regressions.


category: gpa -> gnupg


GnuPG's BTS <gnupg@bugs.g10code.com>
<T1624>


werner lowered the priority of this task from High to Normal.Jan 15 2016, 9:57 AM

I commited an adjusted patch for GnuPG 2.1 (3e50236).

werner closed this task as Resolved.