It probably would have been better to create two issues:
a) Dataloss with Kleo in 2.2.2 (fixed now)
b) crash with gpa

Windows does not allow file names with a '*'. I'm not sure on what level but Its
ok not to handle this case.
I don't expect any problems for internal usage. Keep in mind that this is a
regression, we had wildcard expansion before we made the switch to mingw-w64.

We also don't need this in gpgwrap as gpgwrap just passes the argument on and it
will be expanded in the process itself.

But I actually like the idea to do the wildcard expansion in kleowrap / gpgwrap.
This way it would be contained in Gpg4win and we catch all our "user exposed"
processes. Ok?

I won't do that just for gpg - this would be inconsistent. The wrapper we put
into the PATH directory needs this as well. What about gtk and qt libraries -
they run exe files internally - will the quoting continue to work? A single '*'
in a file name would likely break Enigmail.

Well just gpg would be enough imho as this is by far the most prominent command
line tool.

On the other hand it might be more prudent for us to hack / patch it just in the
gpg4win build to have it enabled globally for all tools we ship so that it is
more consistent. This would mean patching the compiler tough which we tried to
avoid so far.

I would be fine with moving this patch to the version independet gnupg2 patches
in gpg4win as it is kind of a "distribution" option forced upon gpg4win by the
compiler we are currently using.

Werner: If you agree please give a short ping here and I'll move the patch /
close the issue.

Now, shall I add this to gnupg 2.1? To which tools? All or just gpg?

I've changed the category to gpa, adjusted the topic and version to 2.2.3
As you've already described the problem together with GPA here I think this is
better then opening a new bug.

I'll also no longer call this critical as the original data loss problem
(Encrypting files where one has an umlaut -> kleo thinks its a success and
deletes the original) Should be resolved.

The fix in GPA should be fairly easy. Some conversion from native to utf-8 on
input and utf-8 to native on output. So I'm taking this issue.

Werner: Could you please take a look at the patch for gpgtar. I will probably
propose something quite similar for GPA. Not real unicode support but at least
for 8 bit filenames.

I did install gpg4win 2.2.3 which does include the fix for Umlaut. I prefer to
use GPA as frontend for encryption and decryption. For decryption, I did link
.gpg to be always opened with GPA. Unfortunatly if the File or Path to the .gpg
has an Umlaut included the GPA GUI crashes. Only if I use the open button from
GPA the file can be added to the GPA frontend and decrypted. For non-umlaut
files the link to open .gpg works fine. Do you also work on GPA bugs or do I
have to report this under the GPA category?
Thank you & regards,

Patch is included in gpg4win now with a comment that it should be obsolete with
newer mingw versions.

The error is now handled by Kleopatra:

http://commits.kde.org/kdepim/2a58b4cb452cdb132553c2381ce810bbc2606e55

I'm a bit scared of regressions, though as the Input handling is so
"generalized" in kleo that I don't know if I now broke cases where input errors
are expected :-/

Attached is a screenshot how it looks now with a broken gpgtar version. And
files are no longer deleted as the operation is no longer thought to be successful.

D192: 523_0001-Fix-gpgtar-8-bit-encoding-handling-on-Win32.patch

Werner: Please see attached Patch.

I've tested this with kleopatra and it works now to encrypt a folder which has
special characters in the local code page in filenames. And tested it on the
command line.

This should improve the status quo a lot as full utf16 file names are rarer and
gpgtar / gpg4win is not the only tool which has problems with utf16 filenames ;-)

I did not go for proper unicode support because this would mean:

1. Convert all command line arguments from native to utf8. (Or even to expect

them to be utf-16 *brr*)

2. Expect files-from / stdio file names to be utf8 encoded (and update the callers)
3. Use wide char file io functions. Which would add lots of #ifdefs.

But we would have to discuss if we should do 2. also on non-Windows systems.
Currently gpgtar expects local 8 bit encoding there. So tools using gpgtar would
have to convert their arguments differently for Windows.

Thank you for working on this problem. I also use GPA 0.9.4 for Windows as gpg
frontend. Would be great if the fix will make the file encryption working with
umlaut also in GPA and kleopatra.

Thank you & Regards
boehmtho

gpgtar correctly returns an exit code != 0 in case an error happens. This exit
code is eaten by kleopatra so the problem that success is reported on error is
on kleos side.

But gpgtar has the encoding problems. From the kleopatra side it looks ok file
names are handed over to gpgtar in local 8 bit encoding and not in utf-8. And
even directly on the command line it fails.

C:\Users\aheinecke\Desktop>"c:\Program Files\GNU\GnuPG\gpgtar.exe" --skip-crypto
--verbose -o c:\Users\aheinecke\Desktop\test.tar -e fäil.txt
gpgtar: error stat-ing `fΣil.txt': The system cannot find the file specified.

The problem is that internally gpgtar treats argument file names as UTF-8 and
even converts the return value from syscalls like findfirstfile to UTF-8 before
opening them. The open uses the utf8 encoded filename and fails as the file
system usually does not use utf8 file names on Windows.

A workaround would be to convert to ACP (CP_ACP (afaik ACP is correct for
filenames)) instead of converting to UTF-8. But this will not work for ?
but as gpgex and kleopatra already fail earlier on "Non 8 bit representable
characters" this should be acceptable for now.

So two fixes need to be done:

• Use system 8 bit encoding for open in gpgtar. -> Should fix most problems
• Handle the return value of gpgtar correctly and escalate it to the user in

kleopatra.

  -> Should make the rest of the problems "uncritical"

I hope to get this done tomorrow evening.

Got two more reports about this for Windows XP users. So we can safely assume
that this was not just a corner case problem for a broken setup of the Original
Reporter but that it is a real problem.

I'll add a reversion of the commit mentioned fpr 2.2.3

accidentally removed the original reporter from the nosy. Sorry (readded)

T1624 is another issue related to this. GPGex
/ Kleopatra file / folder encrypt does not work with non ASCII characters.

To use --multifile with files on the command line. Such a use will suddenly
break. This is the same as using grep on a large number of files - only rookies
do that.

Fortuntately g_rand_new is only used by the dbus support and by tests. Thus we
can easily revert to the old state.

FWIW: Changing the semantics of an existing function to suddenly claim
"cryptographically secure" is a questionable move. You can't rely on it and the
name of the function indicates that it is a wrapper around the usual rand
function. If a CSRNG is needed, an approriate function needs to be added.

Not sure what you mean by "It is not useful for that purpose" what do you mean
by "It" and "that" in this context.

With regard to complaints there are reports about this that have been active
nearly all the time since this was broken by 2.2.0:
https://wald.intevation.org/forum/message.php?msg_id=2569&group_id=11

This thread triggered your initial opening of this issue.

I agree that wildcard / multifile it is not very important. But I think we
should fix it as it is a small annoyance and it worked with gpg 2.1.0. That it
stopped working was due to our switch to mingw-w64 and not due to a decision
made by us to disable this. So I would see this as a Bug.

Some more threads about this:
https://wald.intevation.org/forum/forum.php?thread_id=1119&forum_id=21&group_id=11
https://wald.intevation.org/forum/message.php?msg_id=3404

P.S.
Yes I dislike the message boards too but they are surprisingly active.

It is not useful for that purpose. This is probably the reason why we received
no complaints about it unless someone uses --multifile with filenames. It is
almost always wrong to do this.

I disagree, all builtin windows tools do globbing so you expect it on the
command line. And as this is default for software that is built with visual
studio and with some versions of mingw it is more likely to see a software that
supports globbing then not.

With regards to quoting problems. I think that users would prefer some problems
instead of just "no". There is no xargs equivalent in Windows by default and so
you have to fall back to really ugly batch scripts that do this in for loops.

Yes one could argue that those users should use a "real shell" on a "real
system" ;-)

Well, it's part of 2.2.2 now, lets see if we get some feedback on this and maybe
for the next release we can revisit the question again if we should enable this
or not.

It would be rather surprising if just one tool does globbing on Windows but all
the otehrs don't do it. It may actually lead to bugs due to broken quoting in
mingw. I recall than many years ago we had a lot of problems with the ever
changing quoting in the globbing implementation.

In any case using globbing with --multifile is not a good idea at all. There is
always a limit on this and thus the proper way of handling this is to use xargs
or whatever the counterpart is on Windows.

Tested this myself again. And I feel comfortable that this is fixed. Still I do
not know why this works now.

Werner can you take a look at the patch I've added for this:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gpg4win.git;a=blob;f=patches/gnupg2-2.0.26/0001-Enable-wildcard-expansion-with-mingw-w64.patch;h=6c5b9f8b0794cf32ab909c0a340f81377dc3d42f;hb=HEAD

Is this something you would consider including in some form in gnupg or should I
move the patch into the generic gnupg patch directory?

Now for 2.2.2 we've switched to a new dedicated buildsystem with a fully updated
debian wheezy whereas the old buildsystem was a debian unstable snapshot with
the same mingw Version: 2.0.3-1
I could not find an upstream bug or any indication that there was a fix for this.

But with beta-37 of 2.2.2 I was able to use wildcards on the command line.
I am out of my wits why this works now.

Emanuel: can you confirm that wildcards now work on the command line so that we
can mention this in the news and maybe look at integrating the patch into gnupg?
I'd like a different pair of eyes on that behavior. Maybe I've just messed up my
testsystem.

I've commited the patch to gpg4win so it will be part of the 2.2.2 release.

Thanks for summing up the other problems. I've added a reference to this issue
to the "Improve encoding handling" point in the backlog:
http://wiki.gnupg.org/Gpg4win/Wishlist

Thanks Andre for the patch!

I managed to build gpg4win with the patch added and I verified that it seems to
solve the problem reported by me and also in Issues 1373 and 1674!

But I'd like to summarize the problems related to the charset / codepage on MS
Windows of which I am aware, as a reminder:

1. incorrect display of GPG 2 output translated into another language (also

reported in Issue 1373 and Issue 1674): fixed by your patch;

2. passphrases (both for secret keys and symmetrical encryption) with non ASCII

characters set using GPG 1.4.18 are considered not valid using GPG 2.0.26 and
vice versa

3. incorrect display of filenames with non ASCII characters (also in Issue 1409)

4. GPG 2.0.26 and 1.4.18 ignore or weirdly comply with --utf8-strings, --no-

utf8-strings or --charset options for utf-8 encoding of encrypted filenames (see
Issue 1409)

5. charset weirdness searching keyserver for some non-ASCII user IDs under non-

UTF-8 locales (see Issue 1514 - although relates to Linux it seems to occur also
on Windows, both CLI and GPA but not Kleopatra)

Hope this will help to improve the great GnuPG :-)

This should be working with gpgOl 1.2.1

As I don't have a test set up I'm setting this to resolved until someone complains.

Good anlysis. Thanks.
Feel free to put it as a patch into gpg4win.
I need to look closer at it because we have have the gettext code also in
libgpg-error. You should also send a DCO for GnuPG.

Pretty picture.

D211: 465_0001-Let-wchar_to_native-convert-to-console-codepage.patch

I've taken a look at this. The problem is that the working conversion code in
jnlib/utf8conv.c is not used on Windows but instead jnlib/w32-gettext.c does
it's own conversion to wchar and then back from wchar to the native codepage
which is simpler and should work.

But the conversion back used the wrong codepage. CP_ACP instead of the codepage
retuned by GetConsoleOutputCP. jnlib/utf8conv.c actually had a comment
explaining why it is neccessary to use GetConsoleOutputCP.

With this changed (see attached patch) I get correct output and can verify /
sign files with non-ascii filenames.

I think gnupg master behaves differently though and I don't have a test setup
for this so the patch is only against STABLE.

Werner any objections into including this patch into GnuPG / Gpg4Win?

Thanks Werner. I had already done a search among the issues using the world "charset", but the 1373 issue was
not displayed.

However the bug occurs more problematically with passphrases: a secret key passphrase with non ASCII characters
set using GPG 1.4.18 for a given key is considered not valid using GPG v 2.0.26 to deal with the same key and
vice versa.
I think this problem may cause a lot of trouble to users.

The bug also occurs using filenames in which there are characters outside the ASCII block. This occurs also
setting LC_MESSAGES=C or LC_MESSAGES=en_US.UTF-8 (i.e. gpg -vvvv --verify C:\Users\Andrea\Downloads\gpg4win-
2.2.2-beta37_è_.exe.sig -> "gpg: assuming signed data in `C:\\Users\\Andrea\\Downloads\\gpg4win-2.2.2-
beta37_Þ_.exe'" in which are also wrongly displayed double backslashes instead of single) and also with gpg
1.4.18 (but without the double backslashes). Luckily this does not prevent the signature verification. It's
similar to T1409.

Some additional information:

the CHCP command executed at the Windows command prompt reports: "Tabella codici attiva: 850" (active codepage
850);

using the -vvvv option with the --verify command, gpg 2.0.26 and 1.4.18 report: "gpg: using character set
`CP850'".

This is known. See the other bug report.

Duplicate of T1373

Ok using the same compiler I've used a year ago I also get the same result I got
a year ago (phew I thought i screwed up the test).

But with the patch this should just work once we change the used mingw-version.
I could not find a bugreport or some other information where and when this is fixed.

I've tested this again with the mingw version that is part of ubuntu and it works.

I've commited a patch to gpg4win to enable this and will check if this also
works with the debian wheezy mingw-crt version.

Thanks for the explanations.

Further analysis showed that the second call to AllowSetForeground window was
blocked by the ForegroundWindowLockTimeout. If you set this timeout to zero
everything worked as expected.

There is a discussion on this under:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/09fa16ba-c6ef-410d-bec2-a99a9b9a98d9/issue-with-foregroundlocktimeout-setforegroundwindow

Which suggests a solution of setting the foregroundlocktimeout to zero before
attempting to set the foreground window. I've found this solution not applicable
for our case as we run into the lock on "AllowSetForegroundWindow" and not when
actually setting the foreground Window.

Another workaround is to call AttachThreadInput on the current foreground thread
call SetForegroundWindow and then detach again. There might be problems with
this approach so it is just used as fallback. But it should resolve this problem
for now.

What I described is done for all pinentries.

Ok, thanks I'll check what happens if

5. The agent thinks that the passphrase entered is too weak
6. The agent starts another pinentry
7. The agent sends "getinfo pid" to the pinentry (this i can see)
8. ?

1. The agent starts pinentry.
2. The agent sends "getinfo pid" to the pinentry"
3. The agents sends an "INQUIRE PINENTRY_LAUNCHED <pid>" to the caller.

4a. If the caller is gpg, gpg calls AllowSetForegroundWindow for the received
pid; gpg has been started via gpgme and gpgme has called ASFW for gpg.
4b. If the caller is gpgsm, it proxies the PINENTRY_LAUNCHED to gpgme which then
calls ASFG for the pid.

The different methods are required because gpg is a one-off process while gpgsm
may be used several times.

Yes Kleopatra uses gpgme for key generation. Still I don't see how gpgme figures
into this sorry. Should gpgme start gpg-agent and ensure that it has the
setforeground window access right?

1. The gpg-agent asks for a passphrase with pinentry -> set foreground window is

allowed.

2. If that passphrase is weak it launches a new pinentry with a confirm dialog

-> set foreground window is not allowed.

And the agent does not have the right to call allowsetforegroundwindow. In the
first case I think this might be because another component gets the PID of the
pinentry and calls allowsetforeground window (have to do further debugging to
check this)

In the second case afaik only gpg-agent is involved and no one sets
allowsetforeground window on the pid of the second pinentry.

BTW, gpgme 1.5.1 has a spawn interface which is better than to code your own in Qt.

It goes all the way back to GPGME via the assuan interface. grep for
_gpgme_allow_set_foreground_window. For GPG the assuan interface is not used.
We do it by passing our internal IOSPAWN_FLAG_ALLOW_SET_FG flag to the
gpgme-w32spawn.c helper which then uses this to call set foreground API.

Does Kleopatra always use gpgme?

I won't call that leaking memory.

With your fix it no longer crashes. But you are leaking memory in case
default_homedir returns allocated memory from read_w32_registry_string or
standard_homedir with a successful call to w32_shgetfolderpath.

As the result should be cached in the static variable this is probably not an
issue..

I just pushed a fix.

The problem with gpgme is that we need to stop the Explorer if we want to update
or uninstall gpgme. Given tha we can't use gpgex with the portable version
anyway, I think it is better to keep the existing code and don't switch to gpgme.

Using gpgme would be nice of course to avoid duplicated functionality (this same
stuff is also implemented in gpgol although without the bug) but remember that
we would then also need a x64 version of gpgme.

The bug is pretty obvious. I consider to rewrite it by calling gpgconf
--list-dirs to get the home directory. This would be simliar to what we do in
gpgme. Or well, we could link to to gpgme to and make use of its higher level
interface.

Thanks for looking at this. If you like I could test your fixes. I currently
have a build setup and a test setup where I can reproduce the crash.

Btw. Maybe I don't understand c++ enough but afaik this just makes no sense and
is broken:

  try { name = ((string) dir) + "\\S.uiserver"; } catch (...) {}

should be rather something like:
if (dir)
name = string(dir) + "\\S.uiserver";

I have some fixes sitting here in my local repo. I need to check them.

Closing this as a duplicate of T1516

Duplicate of T1516