Page MenuHome GnuPG
Create Task
Maniphest T3446

Possible key duplication when using auto-key-retrieve?
Closed, ResolvedPublic
Actions

Description

Today I found out, that my keyring contained various keys of other persons multiple times. That is, the fingerprint of the multiple keys for the person were the same.

As I am using Thunderbird/Enigmail to read Mailinglists I was able to reproduce the error after activating the "auto-key-retrieve" option. It looks like Enigmail does two or more "--verify" calls to GnuPG at once and GnuPG somehow imports a key multiple times then. I was unable to reproduce the error in the Windows commandline until now.

Preparation:

  • Vanilla Thunderbird and Enigmail installation
  • Select GnuPG 2.2.1
  • Input keyserver for automatic import (tested it with: pool.sks-keyservers.net)
  • leave rest of enigmail configuration alone

Steps to reproduce:

  • Empty HomeDir (kill/move all files)
  • Kill gpg-agent and dirmgr if (still) running
  • --list-keys on commandline -> gpg creates keyring and returns 0 keys
  • Select signed email, wait until key has been imported (i.e. msg "signature not verified/checked" is shown or task in taskmanager is gone)
  • --list-keys on commandline -> gpg returns 1 key
  • Select signed email of another person and wait until key import complete
  • --list-keys on commandline -> gpg returns 3 keys !

--> The keyring now contains the same key (checked by fingerprint) two times!

System specs:

  • GnuPG 2.2.1
  • Enigmail 1.9.8.3 (20171001-1439)
  • TB 52.4.0, x86, release
  • Win 8.1 x64

Regards
Alex

Details

Version
2.2.1

Revisions and Commits

rG GnuPG
rG7c73db3d31c6 gpg: Keep a lock during the read-update/insert cycle in import.
rG645f30ad310a gpg: Keep a lock during the read-update/insert cycle in import.

Event Timeline

gp_ast created this task.Oct 11 2017, 5:42 PM
gp_ast created this object in space S1 Public.
• werner triaged this task as Normal priority.Oct 12 2017, 10:18 AM
• werner added a subscriber: • werner.

When Enigmail is running several operations at the same time it is possible that this happens. We would need to take a read lock for the entire time it takes to fetch the key or use other complicated methods to avoid a test/insert race. That would be very inconvenient. The proposed solution is to have just one process to update the keyring.

What happens is this:

  • gpg A verifies signature from key 1
  • gpg A figures that key 1 is missing and starts a key retrieval.
  • gpg B verifies another message also signed by key 1
  • gpg B figures that key 1 is missing and also starts a key retrieval.
  • gpg A gets the key and inserts it into the keyring.
  • gpg B also gets the key and inserts it into the keyring

The insert operation does not check whether the key has already been inserted by another process.

It would be possible to do this but that would still not close the race completely or may lead to a deadlock. Sure, this can all be solved but the gpg processes needs strong synchronization. Thus I consider it better to have a single process to maintain the keys. Such a process would also allow to cache requests for keys etc.

gp_ast added a comment.Oct 12 2017, 2:36 PM

Ok, thanks for the explanation.

Are duplicate keys a problem for GnuPG somehow?
As far as I can see enabling/disabling a key affects all existent keys, deleting affects only one key.
Additionally I can say that a keyring with several duplicate keys slows down GnuPG a fair amount. In my keyring are 18 keypairs and 316 public keys with 96 duplicates (ranging from 2 up to 40 dups per key...). A list-public-keys on a correct keyring is about one third faster. (ca. 230ms compared to ca. 400ms)

Besides this I did not encounter any other problems (or did not notice them).

patrick added a subscriber: patrick.Oct 13 2017, 11:39 AM

Werner, so what do you suggest? Does Enigmail (and any other tool using gpg, and actually also across tools) need to make sure that there are no concurrent calls to gpg of the type that could lead to adding a new key in the keyring?

• werner raised the priority of this task from Normal to Unbreak Now!.Oct 16 2017, 11:40 AM

Looking again at this case I assume this problem is seen more often today because 2.1 started to clean keys during import. That enlarges the time span for the race condition. We clearly need to do something about this in gnupg 2.2.

• werner added a commit: rG645f30ad310a: gpg: Keep a lock during the read-update/insert cycle in import..Oct 19 2017, 9:00 AM
• werner added a comment.Oct 19 2017, 11:56 AM

Fixed in master. Backport to 2.2 pending.

• werner added a commit: rG7c73db3d31c6: gpg: Keep a lock during the read-update/insert cycle in import..Oct 19 2017, 3:10 PM
• werner closed this task as Resolved.Oct 19 2017, 3:11 PM
• werner claimed this task.

Backport to 2.2 done.