D338: 740_0002-scd-map-card-reset-event.patch

Maybe it's more appropriate to map the PSCS_W_CARD_RESET event to the
SW_HOST_CARD_INACTIVE error code which later gets mapped to GPG_ERR_CARD_RESET
error code.

I've attached the patch file. It would make sense to backport this mapping as
well. Right now it's not yet tested.

I found another problem with the smartcard service under windows. Putting
the system into sleep mode and waking it up again creates an 0x80100068
error code (aka PCSC_W_RESET_CARD).

I'll test if it helps to map the RESET_CARD event to the same REMOVE_CARD
event to get the card reactivated after sleep mode.

Logfile:
2015-12-21 22:16:57 scdaemon[10040] DBG: send apdu: c=00 i=CA p1=00 p2=C4
lc=-1 le=256 em=0
2015-12-21 22:16:57 scdaemon[10040] DBG: PCSC_data: 00 CA 00 C4 00
2015-12-21 22:16:57 scdaemon[10040] pcsc_transmit failed: reset card
(0x80100068)
2015-12-21 22:16:57 scdaemon[10040] apdu_send_simple(0) failed: general
error

Okay, Feel free to re-open if you see it again.

I think that this was fixed in:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=d1a97585c5e73fbc7d4cf90e38f76ffc5aea305f

It will be in 2.1.11 and 2.0.30.

Hi Neal, I am not able to reproduce the issue with GnuPG 2.1.10 anymore.

Hello Andre,

I've checked that 2.1.10 still has the problem. So back to you.

You can ping me directly if you need any debug logs or so.

Thanks for helping keep track of all these issues.

Yes this only fixes the problem that has already been fixed in the last Gpg4win
Versions. So that this will be fixed in future gnupg-2.1 versions.

Still to help us better seperate the problems I would like to close this as for
me this bug was about "Wrong encoding in a localized version".

• the more critical "passphrase with non ASCII characters" problem (as reported

only here, see T1691 (andreaerdna on Aug 19 2014, 02:36 AM / Roundup)); does this bug need a
dedicated new Issue to be addressed and solved?

I actually overlooked this in this issue. Can you please open another issue for
that. And add me to the Nosy.

• the "utf-8 encoding of encrypted filenames" / "strange behaviour of --utf8-

strings, --no-utf8-strings and --charset options" (as reported in Issue 1409 ad
probably similar to Gpgtar Issue 1624 / Gpa Issue 2185)

If this problem was still existing with gpg4win this is still a problem.

• the "charset weirdness searching keyserver for some non-ASCII user IDs under

non-UTF-8 locales" (as reported in Issue 1514).

This appears not to be windows specific. Also I think this works except for
cases where the Key in question is problematic. If I search on windows for
emanuel@intevation.de I get the correct Umlauts shown. Might be a Problem though
for characters that are unrepresentable in the 8 Bit codepage.

It sounds great!

So this patch, as the previous one, solves the "incorrect display of GPG 2
output translated into another language" (as reported here and previously also
in Issue 1373 and Issue 1674).

Does this patch solve also the "incorrect display of filenames with non ASCII
characters" (as reported here and previously also in Issue 1409)?

By the way, as I understand, this patch doesn't fix:

• the more critical "passphrase with non ASCII characters" problem (as reported

only here, see T1691 (andreaerdna on Aug 19 2014, 02:36 AM / Roundup)); does this bug need a
dedicated new Issue to be addressed and solved?

• the "utf-8 encoding of encrypted filenames" / "strange behaviour of --utf8-

strings, --no-utf8-strings and --charset options" (as reported in Issue 1409 ad
probably similar to Gpgtar Issue 1624 / Gpa Issue 2185)

• the "charset weirdness searching keyserver for some non-ASCII user IDs under

non-UTF-8 locales" (as reported in Issue 1514).

After some more discussion and testing in the development jabber channel werner
agreed to include this patch. Pushed to libgpg-error with 823e858. So this will
hopefully be part of the first gnupg modern release that will include localization.

Btw. this was patched in Gpg4win for over a year now. So I expect we would have
heard if this caused regressions.

I've opened T2185 for the GPA Problem so i can change the topic here and we
can cleanly close this issue when the gpgtar fix is applied upstream.

We might also want to create a new fix for UTF-16 support in gpgtar once this is
closed. But the attached patch would improve the current situation already a lot.

D210: 737_0001-Fix-windows-8bit-encoding-conversion.patch

Updated Patch against libgpg-error where this code now lives.

Please apply this patch or something similiar.

The problem I can see is that with this code in libgpg-error now GUI
applications may use it which want to get "GUI Native".

Probably better to introduce a new function "wchar_to_console" ? And use it from
GnuPG. Does GPA use that conversion function?

Might be a good time for this now where gnupg master already depends on new
symbols in libgpg-error.

Thank you for your testing.

Your change is pushed with my comment:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=d1a97585c5e73fbc7d4cf90e38f76ffc5aea305f

I'll backport this to GnuPG 2.0.

Here's the logfile with all the errors (guru debug level) vanilla 2.1.10

After some time spending fighting with the build tools of gnupg (cross compile
for windows under debian) I managed to build the installer with my patched
file.

Most important: The most common error thrown is the 0x8010001e
(E_SERVICE_STOPPED) This is the important one. The other error 0x8010001d
(E_NO_SERVICE) is only thrown in the transition from ok to stopped. So only
sometimes.

This was my process:

git clone git://git.gnupg.org/gnupg.git
cd gnupg
git checkout tags/gnupg-2.1.10
./autogen.sh
cat ../0001-scd-Fix-removal-of-unplugged-usb-readers.patch | patch -p1
sed -i -e 's/^SELFCHECK=1/SELFCHECK=0/' build-aux/speedo.mk
make -f build-aux/speedo.mk w32-installer

I've created new logfiles (vanilla 2.1.10 und patched 2.1.10) to show the
difference and confirm that it'S actually working now :-)

I'm okay with signing off the commit. I can test this for Windows 8.1 or 10,
my only problem is that I'm not able to compile gpg for windows right now. Or
are there instructions somewhere on how to achieve this?

Thank you again.

I think that Windows 8 (and later) changed the PC/SC service. The service is
only available when smartcard is there, and after the removal, it returns
PCSC_E_NO_SERVICE error. This is not expected for current code.

I'm applying your patch with my comment like above. Do you agree to put the
line in the commit log?:

   Signed-off-by: Daniel Hoffend <dh@dotlan.net>

I don't have Windows 8 machine. So, I leave this issue as testing.

No, I just installed version 2.1.10 (which included your mentioned fix). But the
error still applies.

In my case the smartcard reader never gets closed, cause the error thrown by the
pcsc/scd gets only mapped to a general_error which does not result in
removing/closing the reader interface.

I've the feeling that we've to take a closer look at the errors thrown (at least
those 2 in my patch). Maybe there're even more possible events.

If you like I can upload the debug log of scdaemon 2.1.10 ... (if that helps).

Somehow I don't have any issues when running linux, this bug applies to windows
only atm. Maybe it's just that windows is throwing different errors or events
compared to linux.

Thank you for the bug report with log.

It could be related to the bug which was just fixed:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commitdiff;h=f42c50dbf00c2e6298ca6830cbe6d36805fa54a3

I'm backporting this to 2.0.x.

After looking at the gnupg 2.0 branch I would say the patch could be applied
to the 2.0 and 2.1 branch to fix the issue in both branches stable/modern
since both version are affected (tested with 2.1.9 and 2.0.29 from gpg2win)

More difficult then I thought.

For PGP/Inline this should currently work. I had the problem that I can't
manipulate the Body in MAPI but over Outlook in the write event this worked.

PGP/Clearsigned support i've disabled for now.

With regards to mime mails:

I could modify / restore the mail there already using old code. The message
is not formed correctly but this looks like just a bug in the revert code.

As it turns out this was totally an understatement ;-) The old revert code can't
have worked. Maybe for S/MIME under some circumstances but otherwise not.

The problem is the main part how Outlook builds the MIME message. Were we have
very limited control about it. Just removing our attachments and leaving the
original MIME attachment leads to a MIME structure like:

<quote>

This is a multipart message in MIME format.

------=_NextPart_000_0000_01D12C53.76E82C90
Content-Type: multipart/alternative;
boundary="----=_NextPart_001_0001_01D12C53.76E82C90"

------=_NextPart_001_0001_01D12C53.76E82C90
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

------=_NextPart_001_0001_01D12C53.76E82C90
Content-Type: text/html;
protocol="application/pgp-encrypted";
boundary="nextPart3167407.zD7nylcVYN";
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-W3CDTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
rmj.rmm.rup.rpr">
<TITLE></TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<BR>

</BODY>
</HTML>
------=_NextPart_001_0001_01D12C53.76E82C90--

------=_NextPart_000_0000_01D12C53.76E82C90
Content-Type: application/pgp-encrypted;
name="Unbenannte Anlage 00001.dat"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="Unbenannte Anlage 00001.dat"

Version: 1
------=_NextPart_000_0000_01D12C53.76E82C90
Content-Type: application/octet-stream;
name="msg.asc"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="msg.asc"

-----BEGIN PGP MESSAGE-----
Version: GnuPG v2

hQEMAx7U8Lxs+8kSAQf/eB4zBTz/VSVBBI+ihh/PSorJ98BRh5earBqF8HjmGZce

<end quote>

This is nothing even an MUA like KMail can handle. And GpgOL can handle this
neither. So if we modify the message we have to do it somehow in a way that
Outlook builds a Mime structure again that users can work with.

As we can actually send MIME messages I looked at the code in mimemaker that
builds a message. Using some tricks from there I was able to recreate a PGP/MIME
mail. But this needs special handling for all our message classes.

Still too buggy to commit. Leaks plaintext and I have at least seen that it led
to a duplicated message once.

I just double checked the code from 2.1 - it looks really okay.
I need to look at the 2.x branch, though.

Modifying the mail in the afterwrite event did not work good. While the
attachment changes were synced to the server Outlook itself didn't reparse the
mail correctly. This let to a weird out of sync situation between MAPI and OOM.
But testing looks like this could work from the Write event indeed. Which would
be even better because we only have one write and we could replace the "Wipe
Message" code completely by just reverting the mail back to the original.

I'm optimistic this can be done. :-)

It's a bit iffy though and might be especially annoying from a performance side
for exchange users. Still it will be better then the Status Quo because you can
still use the mails with other clients.

The trick is not to revert back the message in the Write event, as we have to
work on the OOM in the Write event but in the AfterWrite event where we can work
on MAPI.

I could modify / restore the mail there already using old code. The message is
not formed correctly but this looks like just a bug in the revert code.

Test data from: http://keyserver.borgnet.us/dump/sks-dump-0000.pgp.bz2

In one console window:
mkdir c:\test-issue2135
set GNUPGHOME=c:\test-issue2135
gpg2 --import c:\users\aheinecke\Desktop\sks-dump-0000.pgp

in another:
set GNUPGHOME=c:\test-issue2135
gpg2 -k

Triggers this: (And the error messages also look wrong)

gpg: waiting for lock c:/test-issue2135/pubring.gpg.lock...
gpg: renaming c:/test-issue2135/pubring.gpg' to c:/test-issue2135/pubring.bak'
failed: Permission
denied
gpg: error writing keyring `c:/test-issue2135/pubring.gpg': Permission denied
gpg: key CBB511F4: public key "[User ID not found]" imported
gpg: error reading `c:\\Users\\aheinecke\\Desktop\\sks-dump-0000.pgp':
Permission denied
gpg: import from `c:\\Users\\aheinecke\\Desktop\\sks-dump-0000.pgp' failed:
Permission denied
gpg: Total number processed: 278
gpg: w/o user IDs: 14
gpg: imported: 265 (RSA: 82)
gpg: renaming c:/test-issue2135/pubring.gpg' to c:/test-issue2135/pubring.bak'
failed: Permission
denied
gpg: failed to rebuild keyring cache: Permission denied
gpg: no ultimately trusted keys found

In this case I'm pretty sure that it does not. I check that I can come up with a
testcase that does not involve kleo.

Is Kleopatra messing around with files in ~/.gnupg in any way? IIRC, Kleo
sometimes bypasses gpgme. For example does it open pubring.gpg ?

As Werner stated, this appears to be a user support inquiry, which is better
answered elsewhere. As such, I'm marking this issue as resolved.

gp_ast: have you gotten a chance to try this?

While Werner mentioned that he might have solved this in previous versions I'm
not seeing this in the code. I'll have to test with GPA as it might be that
progress information causes gpgol to temporarily yield the event loop.

Tested this and GPA also blocks Outlook the same way as Kleo. Which was expected
as the blocking happens in GpgOL.

I've disabled the automatic keylisting while an import job is running in
Kleopatra as this is a good idea anyway.

Still this should be fixed although we might want to give it a try with 2.1
instead as it is no longer a hard issue for gpg4win with the workarond in kleo
in place.

The import with 2.0.29 is also very slow on Windows. Over two minutes to import
650 keys while the same import with 2.1.9 on GNU/Linux only takes 20seconds.

Fixed with: b942f73

If GpgOL is deactivated it cleans up after itself and wipes the plaintext from
mails / session encrypts attachments.
If that fails it shows a warning message.

Yes only on Windows. Works for me on GNU/Linux, too.

Only on Windows? A quick check on Unix shows that it works.

1.6.4 has been released

I can't reproduce this problem.

GpgOL tries to talk to Kleopatra through a file in %APPDATA%\gnupg

please make sure the permissions on this folder are correct. If you started
things with Administrator privileges before there might be files in there that
are only accessible for administrators and not with normal user rights.

The GIT master and also the 1.6 branch now has a fix for that problem. A 1.6.4
release sill be done soon.

This problem still occurs with libgcrypt from current master:
libgcrypt 1.7.0-beta259

#0 0x655f24a7 in _gcry_aes_aesni_do_setkey (ctx=0x97f868,

key=0x656621b4 <key_128>

"\350\351\352\353\355\356\357\360\362\363\364\365\367\370\371\372\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004",
<incomplete sequence \343>) at rijndael-aesni.c:117

Ooops - I should know it is my installer :-(
1.6.3.

IIRC, we fixed the alignment in Libgcrypt but I am not sure whether this has
been backported to Libgcrypt 1.6. Which libgcrypt version is used?

Forgot to resolve this as superseeded.

With gcrypt compiled with -mstackrealign as analyzed in T2085 this problem
is also fixed.

So I'm marking this as a duplicate of T2085.

Duplicate of T2085

Backtrace with debug symbols:

(gdb) bt full
#0 0x655ea3e9 in aesni_do_setkey (ctx=0xc6f868,

key=0x6565dc10 <key_128.65421>

"\350\351\352\353\355\356\357\360\362\363\364\365\367\370\371\372\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004",
<incomplete sequence \343>)

at

/home/aheinecke/arbeit/gpg4win/src/gnupg-w32-2.1.7/PLAY/src/libgcrypt/cipher/rijndael.c:248
No locals.
#1 0x655ead8a in do_setkey (ctx=0xc6f868,

key=0x6565dc10 <key_128.65421>

"\350\351\352\353\355\356\357\360\362\363\364\365\367\370\371\372\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004",
<incomplete sequence \343>, keylen=16)

at

/home/aheinecke/arbeit/gpg4win/src/gnupg-w32-2.1.7/PLAY/src/libgcrypt/cipher/rijndael.c:569

initialized = 1
selftest_failed = 0x0
rounds = 10
i = 1
j = 1
r = 1
t = 13813018
rconpointer = 0
KC = 4
hwfeatures = 1472

#2 0x655eb2b1 in rijndael_setkey (context=0xc6f868,

key=0x6565dc10 <key_128.65421>

"\350\351\352\353\355\356\357\360\362\363\364\365\367\370\371\372\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004",
<incomplete sequence \343>, keylen=16)

at

/home/aheinecke/arbeit/gpg4win/src/gnupg-w32-2.1.7/PLAY/src/libgcrypt/cipher/rijndael.c:668

ctx = 0xc6f868

...

info registers
eax 0x6565dc10 1701174288
ecx 0xd25110 13783312
edx 0xc6f868 13039720
ebx 0x0 0
esp 0xc6f760 0xc6f760
ebp 0xc6f760 0xc6f760
esi 0x0 0
edi 0xd24478 13780088
eip 0x655ea3e9 0x655ea3e9 <aesni_do_setkey+31>
eflags 0x10297 [ CF PF AF SF IF RF ]
cs 0x1b 27
ss 0x23 35
ds 0x23 35
es 0x23 35
fs 0x3b 59
gs 0x0 0

disas 0x655ea3e2,0x655ea3ff

Dump of assembler code from 0x655ea3e2 to 0x655ea3ff:

0x655ea3e2 <aesni_do_setkey+24>:     mov    0xc(%ebp),%eax
0x655ea3e5 <aesni_do_setkey+27>:     movdqu (%eax),%xmm1

> 0x655ea3e9 <aesni_do_setkey+31>: movdqa %xmm1,(%edx)

   0x655ea3ed <aesni_do_setkey+35>:     aeskeygenassist $0x1,%xmm1,%xmm2
   0x655ea3f3 <aesni_do_setkey+41>:     pshufd $0xff,%xmm2,%xmm2
   0x655ea3f8 <aesni_do_setkey+46>:     movdqa %xmm1,%xmm3
   0x655ea3fc <aesni_do_setkey+50>:     pslldq $0x4,%xmm3

It appears to be that this is crash is due to the fact that windows uses a 4
Byte stack alignment and the movdqa call expects 16 byte alignment.

I've found some info on this here:
http://www.peterstock.co.uk/games/mingw_sse/

I also confirmed that with "-mstackrealign" the crash no longer happens.

Werner: should we add this globaly to the configure options of gcrypt or do you
have a better fix for this?

This issue seems fixed in gnupg-w32-2.1.7.

...
Or printf debugging was the wrong approach here.

Attaching gdb to the agent led to the following backtrace:

#0 0x655ea3e9 in aesni_do_setkey () from C:\Program
Files\GnuPG\bin\libgcrypt-20.dll
#1 0x655ead8a in do_setkey () from C:\Program Files\GnuPG\bin\libgcrypt-20.dll
#2 0x655eb2b1 in rijndael_setkey () from C:\Program
Files\GnuPG\bin\libgcrypt-20.dll
#3 0x655edadd in selftest_basic_128 () from C:\Program
Files\GnuPG\bin\libgcrypt-20.dll
#4 0x655ede09 in selftest () from C:\Program Files\GnuPG\bin\libgcrypt-20.dll
#5 0x655eabfc in do_setkey () from C:\Program Files\GnuPG\bin\libgcrypt-20.dll
#6 0x655eb2b1 in rijndael_setkey () from C:\Program
Files\GnuPG\bin\libgcrypt-20.dll
#7 0x655cd4ae in cipher_setkey () from C:\Program Files\GnuPG\bin\libgcrypt-20.dll
#8 0x655ce076 in _gcry_cipher_setkey () from C:\Program
Files\GnuPG\bin\libgcrypt-20.dll
#9 0x655c2308 in gcry_cipher_setkey () from C:\Program
Files\GnuPG\bin\libgcrypt-20.dll
#10 0x0041aea8 in agent_protect ()
#11 0x004189a9 in store_key ()
#12 0x0041950b in agent_genkey ()
#13 0x00407a5e in cmd_genkey ()

So I've built libgcrypt again with --disable-aesni-support (Which is also what
gpg4win uses). And the crash goes away.

Surprise. This issue is weird.

Agent calls: hash_passphrase in agent/protect.c:do_encryption
I've added a load of debug output there but this is where it crashes.
I've moved the get_standard_s2k_count out of that call to verify that this is
not he crashing part.

My code looks like this:

  log_debug ("%s:%s: Line: %d", __FILE__, __func__, __LINE__);
  unsigned long s2kcnt = get_standard_s2k_count();
  log_debug ("%s:%s: Line: %d", __FILE__, __func__, __LINE__);
  rc = hash_passphrase (passphrase, GCRY_MD_SHA1,
                        3, iv+2*blklen,
                        s2kcnt,
			key, keylen);
  log_debug ("%s:%s: Line: %d", __FILE__, __func__, __LINE__);

The debug output after the hash_passphrase is not reached. The line before is.

But now this is where it gets weird.

With (debug enhanced):

static int
hash_passphrase (const char *passphrase, int hashalgo,

int s2kmode,
const unsigned char *s2ksalt,
unsigned long s2kcount,
unsigned char *key, size_t keylen)

{

  /* The key derive function does not support a zero length string for
     the passphrase in the S2K modes.  Return a better suited error
     code than GPG_ERR_INV_DATA.  */
  int ret;
  log_debug ("%s:%s: Line: %d", __FILE__, __func__, __LINE__);
  if (!passphrase || !*passphrase)
    return gpg_error (GPG_ERR_NO_PASSPHRASE);
  log_debug ("%s:%s: Line: %d", __FILE__, __func__, __LINE__);
  ret = gcry_kdf_derive (passphrase, strlen (passphrase),
                         s2kmode == 3? GCRY_KDF_ITERSALTED_S2K :
                         s2kmode == 1? GCRY_KDF_SALTED_S2K :
                         s2kmode == 0? GCRY_KDF_SIMPLE_S2K : GCRY_KDF_NONE,
                         hashalgo, s2ksalt, 8, s2kcount,
                         keylen, key);
  log_debug ("%s:%s: Line: %d", __FILE__, __func__, __LINE__);
  log_debug ("ret: %i ", ret);

  return ret;

}

I can see the debug line above the return statement is executed and that it
returns 0! But i don't see the call returning to do_encryption.

The only idea explaining this behavior that i have so far is some kind of stack
corruption where has_passphrase tries to return to an invalid pointer. But i
don't see the problem atm.

This was already reported in T1819 and T2083.

Let's fix it here :-)