What is the current encoding? OEMCP ?

So if gnupg version >= 2.2.22 Kleopatra needs to convert the passed filenames to UTF-8 and pass them with the --utf8-strings option to gpgtar. This needs to be changed in Kleo. -> Assigned to me.

Unicode file names will now also work - see T4083 and T5030

Excellent! thanks for having considered this.

Done for master and 2.2.22 - libgpg-error 1.39 (not yet released) is required for the actual fix.

The options now work as documented. More tests on Window are required and eventually we need to handle non-ascii characters in file names.

It is indeed a limitation. We added these options to support the Kleopatra GUI. To avoid problems with filenames with embedded newlines etc. Kleoptra uses a binary nuls to delimit filenames. And that is what we only support.

I did forget to mention that the key I'm using is 4096 bit long

I was creating a tar archive with 7-Zip on my Windows 10 machine. After the creating was completed I was encrypting the archive like so:

Just to clarify, you were able to decrypt and extract it without error? Which tool did you use to extract the tar archive?

I did encrypt the file myself with the version mentioned above.

Maybe the file was encrypted with a version of gpg4win-3.1.5? We had a serious bug there that sometimes files were corrupted. See: T4332

(BTW, --create is not an option - you meant --encrypt)

That is not easy to fix in a shell script. I would prefer to get rid of gpg-zip or make it a simple wrapper around gpgtar like

FWIW, I added a gpgtar.1

Pushed to 2.2

You could use the --directory option. However< I agree that your suggested changes is less surprising then the current behaviour. Thus I would consider this a bug fix. Can you please apply to 2.2?

To remain compatible with PGP6 we are limited to ustar. If you want to support other archive types, archive first and then encrypt/sign the archive.

@werner, if I understand the description at
https://www.gnu.org/software/tar/manual/html_section/tar_68.html
then ustar would also be able to read "posix" archives.

ustar is the format introduced by PGP 6; also for Windows. This is the only
reason we use it. PGP users demanded that we support that "pgpzip". We can't
drop it.

What I meant by "KArchive" is that we already have all that nice archiving code
in Kleopatra already: https://api.kde.org/frameworks/karchive/html/index.html
To work with standard formats like tar / zip / 7zip etc.

This would get us the included platform abstraction through Qt for stuff like
filenames etc. and we wouldn't have to maintain our own implementations for
these archive formats.

Can you create a new issue with the data "loss" part?

As for the default format:
I think we should use and propose a default format that is mostly compatible
over platforms (and robust in the future). tar "posix" seems to be
such a format. Am not sure how this evaluates for karchive or 7zip.

https://www.gnu.org/software/tar/manual/html_section/tar_68.html gives a good
overview imo.

So yes raising the file name length limit could be problematic with
compatibility and we might have to change more in our implementation to create
formats of a different spec.

From the discussion in the forum it looks like the error was silently discarded
when used in Kleopatra. We need error handling in that case. So I think this is
an Urgent bug as silent discard of archive contents can lead to data loss. So
for me this part is an urgent bug. Actually handling longer filenames is another
issue.

As a sidenote:
Kleopatra already links KArchive for svgz handling so it already contains a good
API for ZIP file creation. I'd like to add that to Kleopatra and make it default
so that the default is not our own error prone tar implementation. (Other tar
implementations also are problematic for windows). In that case we could also
drop the extraction as zip file support is native in the windows file explorer.

And as suggested in the forum entry we should probably also document how to add
7zip support to kleopatra or check for this at runtime and add some 7zip archive
options if it is available.
This should be doable by editing libkleopatrarc but I'd have to check the syntax
myself in the code as its not documented afaik.

I'm unsure about the compatibility issues with using a higher filename-length
limit.

Pushed your patch to master and 2.0. Thanks.

And the testscript I used to test this.

D289: 587_0001-gpgtar-Fix-extracting-files-with-size-512.patch

In gpgtar-extract.c extract_regular

  for (n=0; n < hdr->nrecords;)
    {
      err = read_record (stream, record);
      if (err)
        goto leave;
      n++;
      nbytes = (n < hdr->nrecords)? RECORDSIZE : (hdr->size % RECORDSIZE);

^ this does not work for the last header if hdr->size size is a multiple of 512.
In that case the last record will not be written.

Please check my attached patch which fixes the problem.

To further minimize the test case:

dd if=/dev/urandom of=testfile count=1024 bs=1024
./gpgtar --encrypt --skip-crypto -- testfile > test.tar
./gpgtar --decrypt --skip-crypto -- test.tar
diff ./test.tar_1_/testfile ./testfile

Binary files ./test.tar_1_/testfile and ./testfile differ

last 512 bytes of testfile are missing after extracting it with gpgtar.

Further tracked this problem down to be a gpgtar extraction issue.

Calling gpgtar on the decrypted archive with the same command kleopatra uses:

C:\Users\aheinecke\Desktop>type gpg-archive.tar |"c:\Program
Files\GNU\GnuPG\gpgtar.exe" --openpgp --skip-crypto --set-filename
C:/Users/aheinecke/Desktop/gpg-archive.tar.gpg --decrypt -- -

Produces the corrupted binary. Copying this archive to a GNU/Linux system and
extracting the tarball with GNU Tar produces a valid binary.

Next test on GNU/Linux with:

./gpgtar --version

gpgtar (GnuPG) 2.1.3-beta4

./gpgtar --openpgp --skip-crypto \

--set-filename /home/aheinecke/arbeit/gpg4win/gpg-archive.tar.gpg \
--decrypt -- /home/aheinecke/arbeit/gpg4win/gpg-archive.tar

sha1sum gpg-archive.tar_1_/gpg2.exe

2d387c8fb53d105c31e4cc2ec186e70a365b0c65 gpg-archive.tar_1_/gpg2.exe

tar -fx /home/aheinecke/arbeit/gpg4win/gpg-archive.tar
sha1sum gpg2.exe

54c8c2ec1083943e556255f76ff8f58e623c5b27 gpg2.exe

The second one is correct.

I did install gpg4win 2.2.3 which does include the fix for Umlaut. I prefer to
use GPA as frontend for encryption and decryption. For decryption, I did link
.gpg to be always opened with GPA. Unfortunatly if the File or Path to the .gpg
has an Umlaut included the GPA GUI crashes. Only if I use the open button from
GPA the file can be added to the GPA frontend and decrypted. For non-umlaut
files the link to open .gpg works fine. Do you also work on GPA bugs or do I
have to report this under the GPA category?
Thank you & regards,

The error is now handled by Kleopatra:

http://commits.kde.org/kdepim/2a58b4cb452cdb132553c2381ce810bbc2606e55

I'm a bit scared of regressions, though as the Input handling is so
"generalized" in kleo that I don't know if I now broke cases where input errors
are expected :-/

Attached is a screenshot how it looks now with a broken gpgtar version. And
files are no longer deleted as the operation is no longer thought to be successful.

D192: 523_0001-Fix-gpgtar-8-bit-encoding-handling-on-Win32.patch

Werner: Please see attached Patch.

I've tested this with kleopatra and it works now to encrypt a folder which has
special characters in the local code page in filenames. And tested it on the
command line.

This should improve the status quo a lot as full utf16 file names are rarer and
gpgtar / gpg4win is not the only tool which has problems with utf16 filenames ;-)

I did not go for proper unicode support because this would mean:

1. Convert all command line arguments from native to utf8. (Or even to expect

them to be utf-16 *brr*)

2. Expect files-from / stdio file names to be utf8 encoded (and update the callers)
3. Use wide char file io functions. Which would add lots of #ifdefs.

But we would have to discuss if we should do 2. also on non-Windows systems.
Currently gpgtar expects local 8 bit encoding there. So tools using gpgtar would
have to convert their arguments differently for Windows.

Thank you for working on this problem. I also use GPA 0.9.4 for Windows as gpg
frontend. Would be great if the fix will make the file encryption working with
umlaut also in GPA and kleopatra.

Thank you & Regards
boehmtho

gpgtar correctly returns an exit code != 0 in case an error happens. This exit
code is eaten by kleopatra so the problem that success is reported on error is
on kleos side.

But gpgtar has the encoding problems. From the kleopatra side it looks ok file
names are handed over to gpgtar in local 8 bit encoding and not in utf-8. And
even directly on the command line it fails.

C:\Users\aheinecke\Desktop>"c:\Program Files\GNU\GnuPG\gpgtar.exe" --skip-crypto
--verbose -o c:\Users\aheinecke\Desktop\test.tar -e fäil.txt
gpgtar: error stat-ing `fΣil.txt': The system cannot find the file specified.

The problem is that internally gpgtar treats argument file names as UTF-8 and
even converts the return value from syscalls like findfirstfile to UTF-8 before
opening them. The open uses the utf8 encoded filename and fails as the file
system usually does not use utf8 file names on Windows.

A workaround would be to convert to ACP (CP_ACP (afaik ACP is correct for
filenames)) instead of converting to UTF-8. But this will not work for ?
but as gpgex and kleopatra already fail earlier on "Non 8 bit representable
characters" this should be acceptable for now.

So two fixes need to be done:

• Use system 8 bit encoding for open in gpgtar. -> Should fix most problems
• Handle the return value of gpgtar correctly and escalate it to the user in

kleopatra.

  -> Should make the rest of the problems "uncritical"

I hope to get this done tomorrow evening.

accidentally removed the original reporter from the nosy. Sorry (readded)