Fixed in
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=b30c15bf7c5336c4abb1f9dcd974cd77ba6c61a7

backported by dkg with commit 0c3d764 for 1.4.19

I'm considering fixing this.

Fixed with rev. b879f5b

I've implemented this in fc010b6. If you get a chance to test it, I'd
appreciate any feedback! Thanks!

This is a bug and was fixed in 2e4e10c1. As you correctly observe, it only
impacts fingerprints and thus your workaround is good. Sorry about that!

To do writes, we use a copy-update-move scheme. Thus, all updates are atomic.
A read fopen()s the keyring or keybox, seeks and reads. If an update occurs
between the seek and read, the reader will see the old version: fopen is
associated with the inode, not the filename:

  reader                writer
  -------               -------
  fopen("keyring.pub")
  seek(fp)
                         cp("keyring.pub", "keyring.pub~")
                         update("keyring.pub~")
                         mv("keyring.pub~", "keyring.pub")
  read(fp)

Thus, writers don't interfere with readers.

We need to lock the underlying file for updates to avoid the case in which two
updates occur nearly simultaneously, but only one is saved. (Also, since the
updates occur in keyring.pub~, we need to ensure exclusive access to that file.)

  writer1                           writer2
  -------                           -------
  cp("keyring.pub", "keyring.pub~")
  update("keyring.pub~")
                                    cp("keyring.pub", "keyring.pub~")
                                    update("keyring.pub~")
  mv("keyring.pub~", "keyring.pub")
                                    mv("keyring.pub~", "keyring.pub")

In the above case, writer1's update is lost. (Note: it could be worse: if both
update keyring.pub~ simultaneously, there could be corruption.)

The bug that I'm describing below only has to do with the key present cache,
which becomes inconsistent, because we don't track external writes.

It is base64 trimmed the last '='.

Introducing new specifier, say %f, would be good, while keeping %F as is.
%f includes the hash algorithm string as SSH does.

I think that current lock/unlock mechanism is only for mutual exclusion between
multiple writers. I mean, lock/unlock is done to avoid inconsistency caused by
multiple writers.

It seems that we forget to implement mutual exclusion between writers and
readers, as Neal described.

Before 2.1.10, the write access was limited to specific interactive usage
patterns and it didn't cause major problems (it caused rarely if happened).
Now, I think that we should implement mutual exclusion between readers and writers.

D342: 738_0001-gpg-Fix-TOCTTOU-when-updating-keyblocks.patch

I've attached a fix that does a very small and straightforward modification to
keydb_update_keyblock, which fixes this problem for both the keyring and keybox.

I confirm that I'm not able to reproduce T2176 (guilhem on Dec 11 2015, 02:21 PM / Roundup) nor T2176 (guilhem on Dec 11 2015, 01:07 PM / Roundup) with 4ffe44c, so
I'm changing the issue to “resolved”. Thanks for the prompt action!

My proposed solution is to change keydb_update_keyblock. We don't actually need
to touch the keybox or keyring code.

By the new behavior, I guess you mean getting an error when deleting a key, but
it fails because another process already deleted it. If something like this
were to current occur, then we'd end up with silent corruption. So, it's not
clear how this new behavior would introduce new behavior that could raise problems.

atomicly here mean that the update/insert functions locate an possibly existing
key using the fingerprint while holding the lock.

Anyway, to really fix that we need a daemon taking control of all keys - a task
for 2.3,

I was aware of that problem but always wondered why I never noticed such a case.
Your analysis is correct and explains the problem. The locking of the keyblock
does not help here (it was introduced only a few years ago).

Instead of making use of found.offset and fix that with your suggested trick we
should not use the offset at all but let the update and insert functions handle
it atomicly - this may result in an insert/update error (e.g. if another process
inserted/deleted the key) but that is an expected outcome if two processes
manipulate the same key.

This should not be fixed for the old keyring format but only for the keybox format:

1. The keyring format is deprecated
2. This introduces a new behaviour and may raise other problems.

If you want to fix that, please do that in a new branch.

This should be fixed in 2e4e10c. Please let me know if it works for you (and
feel free to mark this bug as resolved if it does).

I found the bug. I'll try to create a patch soon. Thanks for reporting this.

This is a good suggestion. Thanks.

Just to be clear: you tested with, say, a long key id, and the output was fine?
In other words, the problem only occurs when specifying a fingerprint?

I think that this was fixed in:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=d1a97585c5e73fbc7d4cf90e38f76ffc5aea305f

It will be in 2.1.11 and 2.0.30.

I confirmed that this is fixed in 2.0 and 2.1.

Thank you. There is no reason. It is fixed in:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=4ee881bff4c8fdfa4b3b7a4b7afab611471e97f1

Thank you for your audit.

It ignores the calculated value if it detects failure of gcry_pk_verify.
This is now a kind of standard practice to avoid possible attacks.

Here is a reference:
https://securityblog.redhat.com/2015/09/02/factoring-rsa-keys-with-tls-perfect-forward-secrecy/

For my case with OpenPGPcard, the patch fixed the problem of wrong fingerprint
computation. Please test with the patch.

Sorry for my mistake for reading your post. I considered it would be the case
for m, but I also fixed the case for e, the exponent.
Here, I reproduce the problem with OpenPGPcard (while it only occurs 1/256 with
Gnuk Token).
I confirmed that original OpenPGPcard returns e as four bytes 00 01 00 01 with
0x00 in front. This causes 100% failure for fingerprint computation.
I'm going to test the patch with OpenPGPcard. (I'm now installing newer
libgpg-error, to build master of GnuPG.)

Note the corruption that occurs is rather subtle. It occurs silently, because
copy_some_packets doesn't throw an error if the next packet to process doesn't
start on STOPOFF, but continues until the offset of the next packet to process
is at *or exceeds* STOPOFF.

Imagine that we have a keyblock A at offset 0 and a second keyblock B at offset
100 with 2 packets:

• A
• B
• The first gpg process does a search for the key at offset 100
• A second process looks up and updates the key block (A') at offset 0 such

that it now has a length of 150 and 4 packets after offset 100.

3. The initial process "updates" B to B'. hd->found.offset now point into the

middle of A'. In keyring.c:do_copy, the first 100 bytes plus any bytes required
to complete the last packet are copied (by copy_some_packets). The next 2
packets are deleted (skip_some_packets) and the new keyblock is inserted. We
now have the following:

• 100+ bytes of A'
• B'
• Last two packets of A'
• B

And B appears to be duplicated.

Note: there is also a TOCTTOU bug for keydb_search / keydb_get_keyblock.

Hi Neal, I am not able to reproduce the issue with GnuPG 2.1.10 anymore.

Hello Andre,

I've checked that 2.1.10 still has the problem. So back to you.

You can ping me directly if you need any debug logs or so.

gniibe: its not one failure in 248. It was 248 failures in 248 tries...

werner: I had to downgrade to have a working system. I hope I'll find time to
reproduce this this week

It seems to be base64:

% ssh -V
OpenSSH_7.1p1 Debian-3, OpenSSL 1.0.2e 3 Dec 2015
% ssh-keygen -l -f .ssh/known_hosts -F playfair.gnupg.org -E md5 -q
playfair.gnupg.org RSA MD5:cc:dd:46:8e:ef:3d:d9:34:97:f8:b8:5a:59:51:80:4a
% ssh-keygen -l -f .ssh/known_hosts -F playfair.gnupg.org -E sha256 -q
playfair.gnupg.org RSA SHA256:KCh034SD0rMKqCkJbdH2wx354s1278tqt9F+xb5cidg

Thank you for the bug report. The ratio of 1 failure among 248 made me a great
hint to locate the bug.

I think that it is fingerprint computation bug, which is fixed here:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=d40975cbe8ff86fcc4a1b4963fdffc66ddee85ce

Thanks for helping keep track of all these issues.

Yes this only fixes the problem that has already been fixed in the last Gpg4win
Versions. So that this will be fixed in future gnupg-2.1 versions.

Still to help us better seperate the problems I would like to close this as for
me this bug was about "Wrong encoding in a localized version".

• the more critical "passphrase with non ASCII characters" problem (as reported

only here, see T1691 (andreaerdna on Aug 19 2014, 02:36 AM / Roundup)); does this bug need a
dedicated new Issue to be addressed and solved?

I actually overlooked this in this issue. Can you please open another issue for
that. And add me to the Nosy.

• the "utf-8 encoding of encrypted filenames" / "strange behaviour of --utf8-

strings, --no-utf8-strings and --charset options" (as reported in Issue 1409 ad
probably similar to Gpgtar Issue 1624 / Gpa Issue 2185)

If this problem was still existing with gpg4win this is still a problem.

• the "charset weirdness searching keyserver for some non-ASCII user IDs under

non-UTF-8 locales" (as reported in Issue 1514).

This appears not to be windows specific. Also I think this works except for
cases where the Key in question is problematic. If I search on windows for
emanuel@intevation.de I get the correct Umlauts shown. Might be a Problem though
for characters that are unrepresentable in the 8 Bit codepage.

I'll try to look into that.

It sounds great!

So this patch, as the previous one, solves the "incorrect display of GPG 2
output translated into another language" (as reported here and previously also
in Issue 1373 and Issue 1674).

Does this patch solve also the "incorrect display of filenames with non ASCII
characters" (as reported here and previously also in Issue 1409)?

By the way, as I understand, this patch doesn't fix:

• the more critical "passphrase with non ASCII characters" problem (as reported

only here, see T1691 (andreaerdna on Aug 19 2014, 02:36 AM / Roundup)); does this bug need a
dedicated new Issue to be addressed and solved?

• the "utf-8 encoding of encrypted filenames" / "strange behaviour of --utf8-

strings, --no-utf8-strings and --charset options" (as reported in Issue 1409 ad
probably similar to Gpgtar Issue 1624 / Gpa Issue 2185)

• the "charset weirdness searching keyserver for some non-ASCII user IDs under

non-UTF-8 locales" (as reported in Issue 1514).

Please do not open another bug but comment on the very same bug you posted a few
days ago (issue2166).

In any case, this is a question and not appropriate for a bug tracker. Use one
of the mailing lists for such questions or see https://gnupg.org/service.html
for commercial support offers.

Duplicate of T2166

After some more discussion and testing in the development jabber channel werner
agreed to include this patch. Pushed to libgpg-error with 823e858. So this will
hopefully be part of the first gnupg modern release that will include localization.

D210: 737_0001-Fix-windows-8bit-encoding-conversion.patch

Updated Patch against libgpg-error where this code now lives.

Please apply this patch or something similiar.

The problem I can see is that with this code in libgpg-error now GUI
applications may use it which want to get "GUI Native".

Probably better to introduce a new function "wchar_to_console" ? And use it from
GnuPG. Does GPA use that conversion function?

Might be a good time for this now where gnupg master already depends on new
symbols in libgpg-error.

Thank you for your testing.

Your change is pushed with my comment:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=d1a97585c5e73fbc7d4cf90e38f76ffc5aea305f

I'll backport this to GnuPG 2.0.

Here's the logfile with all the errors (guru debug level) vanilla 2.1.10

After some time spending fighting with the build tools of gnupg (cross compile
for windows under debian) I managed to build the installer with my patched
file.

Most important: The most common error thrown is the 0x8010001e
(E_SERVICE_STOPPED) This is the important one. The other error 0x8010001d
(E_NO_SERVICE) is only thrown in the transition from ok to stopped. So only
sometimes.

This was my process:

git clone git://git.gnupg.org/gnupg.git
cd gnupg
git checkout tags/gnupg-2.1.10
./autogen.sh
cat ../0001-scd-Fix-removal-of-unplugged-usb-readers.patch | patch -p1
sed -i -e 's/^SELFCHECK=1/SELFCHECK=0/' build-aux/speedo.mk
make -f build-aux/speedo.mk w32-installer

I've created new logfiles (vanilla 2.1.10 und patched 2.1.10) to show the
difference and confirm that it'S actually working now :-)

I'm okay with signing off the commit. I can test this for Windows 8.1 or 10,
my only problem is that I'm not able to compile gpg for windows right now. Or
are there instructions somewhere on how to achieve this?

Thank you again.

I think that Windows 8 (and later) changed the PC/SC service. The service is
only available when smartcard is there, and after the removal, it returns
PCSC_E_NO_SERVICE error. This is not expected for current code.

I'm applying your patch with my comment like above. Do you agree to put the
line in the commit log?:

   Signed-off-by: Daniel Hoffend <dh@dotlan.net>

I don't have Windows 8 machine. So, I leave this issue as testing.

Closing, I assume it's the same bug of 2112, which was fixed.

No, I just installed version 2.1.10 (which included your mentioned fix). But the
error still applies.

In my case the smartcard reader never gets closed, cause the error thrown by the
pcsc/scd gets only mapped to a general_error which does not result in
removing/closing the reader interface.

I've the feeling that we've to take a closer look at the errors thrown (at least
those 2 in my patch). Maybe there're even more possible events.

If you like I can upload the debug log of scdaemon 2.1.10 ... (if that helps).

Somehow I don't have any issues when running linux, this bug applies to windows
only atm. Maybe it's just that windows is throwing different errors or events
compared to linux.

Thank you for the bug report with log.

It could be related to the bug which was just fixed:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commitdiff;h=f42c50dbf00c2e6298ca6830cbe6d36805fa54a3

I'm backporting this to 2.0.x.