Keyring locking on Windows broken
Closed, ResolvedPublic

Description

Afaik this is yet another case of problems created (exposed) by kleopatra's
"Each time something changed in the GNUPGHOME in the last 5 seconds start a
keylist job" behavior. But this works on a GNU/Linux system and I think gnupg
should be robust enough to survive this.

To reproduce:

  • I've exported my pubring with ~650 keys. (gpg2 --export -a > full-export.asc)
  • On Windows I've removed the existing pubring.
  • Start Kleopatra
  • On the command line "gpg2 --import Desktop\full-export.asc"

Output:

<many import lines with no errors>
gpg: 100 keys processed so far
< 5 more keys imported >
gpg: waiting for lock C:/Users/aheinecke/AppData/Roaming/gnupg/pubring.gpg.lock...
gpg: waiting for lock C:/Users/aheinecke/AppData/Roaming/gnupg/pubring.gpg.lock...
< 15 more keys imported >
gpg: conversion from utf-8' to CP437' failed: Illegal byte sequence
gpg: key 2806AF71: public key "\xd0\xa7\xd0\xb0\xd1\x81\xd0\xbb\xd0\xb0\xd0\xb2
\xd0\x98\xd0\xbb\xd0\xb8\xd1\x9b <caslav.ilic@gmx.net>" imported
gpg: key E69A226A: public key "Bogomil "Bogo" Shopov <shopov@kolabsys.com>" imported
gpg: renaming `C:/Users/aheinecke/AppData/Roaming/gnupg/pubring.gpg' to
`C:/Users/aheinecke/AppData/Roaming/gnupg/pubring.bak' failed: Permission denied
gpg: error writing keyring
`C:/Users/aheinecke/AppData/Roaming/gnupg/pubring.gpg': Permission denied
gpg: key 96D94189: public key "[User ID not found]" imported
gpg: error reading `Desktop\\full-export.asc': Permission denied
gpg: no valid OpenPGP data found.
gpg: import from `Desktop\\full-export.asc' failed: Permission denied
gpg: Total number processed: 122
gpg: imported: 123 (RSA: 25)

The utf-8 conversion error appears to be a red herring when I run the same test
without starting Kleopatra first I get the "conversion failed" error but the
import completes. So I think the problem is triggered by Kleopatra running a
keylist during the import.

I've not tested this yet with 2.1.

Details

Version
2.1.10

I've disabled the automatic keylisting while an import job is running in
Kleopatra as this is a good idea anyway.

Still this should be fixed although we might want to give it a try with 2.1
instead as it is no longer a hard issue for gpg4win with the workarond in kleo
in place.

The import with 2.0.29 is also very slow on Windows. Over two minutes to import
650 keys while the same import with 2.1.9 on GNU/Linux only takes 20seconds.

Is Kleopatra messing around with files in ~/.gnupg in any way? IIRC, Kleo
sometimes bypasses gpgme. For example does it open pubring.gpg ?

In this case I'm pretty sure that it does not. I check that I can come up with a
testcase that does not involve kleo.

Test data from: http://keyserver.borgnet.us/dump/sks-dump-0000.pgp.bz2

In one console window:
mkdir c:\test-issue2135
set GNUPGHOME=c:\test-issue2135
gpg2 --import c:\users\aheinecke\Desktop\sks-dump-0000.pgp

in another:
set GNUPGHOME=c:\test-issue2135
gpg2 -k

Triggers this: (And the error messages also look wrong)

gpg: waiting for lock c:/test-issue2135/pubring.gpg.lock...
gpg: renaming c:/test-issue2135/pubring.gpg' to c:/test-issue2135/pubring.bak'
failed: Permission
denied
gpg: error writing keyring `c:/test-issue2135/pubring.gpg': Permission denied
gpg: key CBB511F4: public key "[User ID not found]" imported
gpg: error reading `c:\\Users\\aheinecke\\Desktop\\sks-dump-0000.pgp':
Permission denied
gpg: import from `c:\\Users\\aheinecke\\Desktop\\sks-dump-0000.pgp' failed:
Permission denied
gpg: Total number processed: 278
gpg: w/o user IDs: 14
gpg: imported: 265 (RSA: 82)
gpg: renaming c:/test-issue2135/pubring.gpg' to c:/test-issue2135/pubring.bak'
failed: Permission
denied
gpg: failed to rebuild keyring cache: Permission denied
gpg: no ultimately trusted keys found

I just double checked the code from 2.1 - it looks really okay.
I need to look at the 2.x branch, though.

aheinecke claimed this task.Dec 9 2015, 8:54 PM

I've checked that 2.1.10 still has the problem. So back to you.

You can ping me directly if you need any debug logs or so.

aheinecke reassigned this task from aheinecke to werner.Dec 14 2015, 12:27 PM
aheinecke changed Version from 2.0.29 to 2.1.10.

Yesterday I had a failed Keygeneration while GpgOL's certificate selection
dialog in Kleopatra was open. Tried again and it worked. I did not get Debug
output but the pattern suggests to me that the Certificate selection dialog
looked for changes in the pubring while generating a key and the locking broke
again.

This problem is rising in my priority of Windows Issues as it causes random
failures. There is also a load of similar reports on various channels to be
found through google https://www.google.at/search?q=pubring.bak

aheinecke raised the priority of this task from Normal to High.Jan 5 2016, 11:03 AM
aheinecke renamed this task from Bulk key import fails on Windows if Kleopatra is running to Keyring locking on Windows broken.

Renamed the issue to my current understanding of this problem. Locking on
Windows does not work properly.

werner added a comment.Jan 5 2016, 2:33 PM

Do you mean Kleo opens the pubring file itself?

No Kleopatra does not open the pubring. Let's leave kleopatra out of this.

This bug is about multiple GnuPG processes that conflict with each other. See
msg7466 for an example.

I have pushed chnages to master to fix this problem. One drawback is that
during an import another process "gpg -k" may rarely see no keys at all. A full
fix would either require that we lock the keyrings during all read-only
operations, which would severely hit on the performance of all common
operations, or change the whole system to use a new key access daemon.

If this works the changes need to be backported to 2.0.

werner added a project: In Progress.

Tested this with keybox and it appears to be working. When running a keylist
while importing the import holds for a bit and continues after the keylist.
Not tested this with keyring yet.

Okay, so I can backport this to 2.0 ?

werner lowered the priority of this task from High to Normal.Feb 24 2016, 2:24 PM

I've tested it with pubring now too and it works.
Justus mentioned in jabber that he noticed some more errors after this patch in
the scheme tests. I've not tried them.

@werner The backport to 2.0 didn't happen, I think. Is this still relevant. @justus Do you recall any more problems in the tests?

No I don't recall any such problems, sorry.

werner closed this task as Resolved.Jul 12 2017, 7:03 PM

Given that 2.0 reaches EOL in 6 months and the bug has been here for ages, I won't backport it to 2.0 anymore.