I've implemented this in fc010b6. If you get a chance to test it, I'd
appreciate any feedback! Thanks!

This is a bug and was fixed in 2e4e10c1. As you correctly observe, it only
impacts fingerprints and thus your workaround is good. Sorry about that!

To do writes, we use a copy-update-move scheme. Thus, all updates are atomic.
A read fopen()s the keyring or keybox, seeks and reads. If an update occurs
between the seek and read, the reader will see the old version: fopen is
associated with the inode, not the filename:

  reader                writer
  -------               -------
  fopen("keyring.pub")
  seek(fp)
                         cp("keyring.pub", "keyring.pub~")
                         update("keyring.pub~")
                         mv("keyring.pub~", "keyring.pub")
  read(fp)

Thus, writers don't interfere with readers.

We need to lock the underlying file for updates to avoid the case in which two
updates occur nearly simultaneously, but only one is saved. (Also, since the
updates occur in keyring.pub~, we need to ensure exclusive access to that file.)

  writer1                           writer2
  -------                           -------
  cp("keyring.pub", "keyring.pub~")
  update("keyring.pub~")
                                    cp("keyring.pub", "keyring.pub~")
                                    update("keyring.pub~")
  mv("keyring.pub~", "keyring.pub")
                                    mv("keyring.pub~", "keyring.pub")

In the above case, writer1's update is lost. (Note: it could be worse: if both
update keyring.pub~ simultaneously, there could be corruption.)

The bug that I'm describing below only has to do with the key present cache,
which becomes inconsistent, because we don't track external writes.

D342: 738_0001-gpg-Fix-TOCTTOU-when-updating-keyblocks.patch

I've attached a fix that does a very small and straightforward modification to
keydb_update_keyblock, which fixes this problem for both the keyring and keybox.

My proposed solution is to change keydb_update_keyblock. We don't actually need
to touch the keybox or keyring code.

By the new behavior, I guess you mean getting an error when deleting a key, but
it fails because another process already deleted it. If something like this
were to current occur, then we'd end up with silent corruption. So, it's not
clear how this new behavior would introduce new behavior that could raise problems.

This should be fixed in 2e4e10c. Please let me know if it works for you (and
feel free to mark this bug as resolved if it does).

This should be fixed in e573e6. If you get a chance to test it and can report
back here, I'd appreciate it. Thanks!

I found the bug. I'll try to create a patch soon. Thanks for reporting this.

This is a good suggestion. Thanks.

Thanks for reporting the issue and for following up. For future reference, if
you find that a bug is not a bug, please also change its status to resolved.
Thanks!

Just to be clear: you tested with, say, a long key id, and the output was fine?
In other words, the problem only occurs when specifying a fingerprint?

Note the corruption that occurs is rather subtle. It occurs silently, because
copy_some_packets doesn't throw an error if the next packet to process doesn't
start on STOPOFF, but continues until the offset of the next packet to process
is at *or exceeds* STOPOFF.

Imagine that we have a keyblock A at offset 0 and a second keyblock B at offset
100 with 2 packets:

• A
• B
• The first gpg process does a search for the key at offset 100
• A second process looks up and updates the key block (A') at offset 0 such

that it now has a length of 150 and 4 packets after offset 100.

3. The initial process "updates" B to B'. hd->found.offset now point into the

middle of A'. In keyring.c:do_copy, the first 100 bytes plus any bytes required
to complete the last packet are copied (by copy_some_packets). The next 2
packets are deleted (skip_some_packets) and the new keyblock is inserted. We
now have the following:

• 100+ bytes of A'
• B'
• Last two packets of A'
• B

And B appears to be duplicated.

Note: there is also a TOCTTOU bug for keydb_search / keydb_get_keyblock.

I can't reproduce this. Normally, gpg uses EE37CF96 to sign. When I add
"default-key 4F43C989" to my gpg.conf file, it uses the indicated key:

  $ echo | gpg2 -s | gpg2
  Please enter the passphrase to unlock the OpenPGP secret key:
  "Testing (insecure!)"
  1024-bit RSA key, ID EE37CF96,
  created 2015-09-18.
  
   Passphrase: 
  
  gpg: Signature made Fri 11 Dec 2015 12:49:16 PM CET using RSA key ID EE37CF96
  gpg: Good signature from "Testing (insecure!)" [ultimate]

Add a default-key to gpg.conf:

$ echo | gpg2 -s | gpg2

gpg: using "4F43C989" as default secret key

Please enter the passphrase to unlock the OpenPGP secret key:

"Testing <testing@testing.com>"

1024-bit RSA key, ID 4F43C989,

created 2015-11-17.



 Passphrase: 



gpg: Signature made Fri 11 Dec 2015 12:49:35 PM CET using RSA key ID 4F43C989

6dc37c5 fixes the ambiguous key specification error. The problem is that FPR2
actually appears in your keyring twice! Are you use a keyring or a keybox? Use:

  $ gpg2 --debug=lookup -k FPR2
  gpg: DBG: check_user_ids: Checking --hidden-encrypt-to=FPR2
  gpg: DBG: keydb_search: 1 search descriptions:
  gpg: DBG: keydb_search   0: LONG_KID: 'FPR2'
  gpg: DBG: keydb_search: searching keyring (resource 0 of 1)

keyring, not keybox

gpg: DBG: keyring_search: need_uid = 0; need_words = 0; need_keyid = 1;

need_fpr = 0; any_skip = 0

  gpg: DBG: keyring_search: initializing offset table. (need_keyid: 1 => 1)
  gpg: DBG: keyring_search: searching from start of resource.
  gpg: DBG: keyring_search: packet starting at offset 3357784 matched descriptor 0

first match

gpg: DBG: keyring_search: returing success
gpg: DBG: keydb_search: searched keyring (resource 0 of 1) => Success
gpg: DBG: check_user_ids: Checking if --hidden-encrypt-to='FPR2' is ambiguous

or there are dups

gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: LONG_KID: 'FPR2'
gpg: DBG: keydb_search: searching keyring (resource 0 of 1)
gpg: DBG: keyring_search: need_uid = 0; need_words = 0; need_keyid = 1;

need_fpr = 0; any_skip = 0

  gpg: DBG: keyring_search: initializing offset table. (need_keyid: 1 => 1)
  gpg: DBG: keyring_search: not searching from start of resource.
  gpg: DBG: keyring_search: packet starting at offset 3562559 matched descriptor 0

second match

gpg: DBG: keyring_search: returing success
gpg: DBG: keydb_search: searched keyring (resource 0 of 1) => Success
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: LONG_KID: 'FPR2'
gpg: DBG: keydb_search: searching keyring (resource 0 of 1)
gpg: DBG: keyring_search: need_uid = 0; need_words = 0; need_keyid = 1;

need_fpr = 0; any_skip = 0

gpg: DBG: keyring_search: initializing offset table. (need_keyid: 1 => 1)
gpg: DBG: keyring_search: not searching from start of resource.
gpg: DBG: keyring_search: no matches (EOF)
gpg: DBG: keydb_search: searched keyring (resource 0 of 1) => EOF
gpg: Warning: FPR2 appears in the keyring 2 times.

I wonder if it is an encoding problem.

When this feature becomes available, then we should probably disable
"gtk-entry-password-hint-timeout". See the following Debian bug report for details:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=801757

werner: Are you suggesting that there are programs that use an ambiguous
specification in order to delete just one key? I'm trying to imagine how this
is not undesirable behavior. Do you have any examples of this?

I'm sorry, but this is a bug. --delete-key needs to indicate to the user that
the key specification is ambiguous. Further, in non-batch mode, it should
iterate over all of the keys that match the specification and no such take the
first one.

Here's is the status quo:

$ gpg2 --keyserver pool.sks-keyservers.net --recv-key A56E15A3

gpg: key A56E15A3: public key "Hubert Figuiere <hub@figuiere.net>" imported

gpg: key A56E15A3: public key "Pedro R. Fernandez (GNU/Linux User #000.204.555

http://counter.li.org/) <prafer@eresmas.com>" imported

  $ gpg2 --keyserver pool.sks-keyservers.net --delete-key A56E15A3
  pub  dsa1024/A56E15A3 2002-01-30 Hubert Figuiere <hub@figuiere.net>

  Delete this key from the keyring? (y/N) n
  $ gpg2 --keyserver pool.sks-keyservers.net --delete-key A56E15A3
  gpg (GnuPG) 2.1.10-beta159; Copyright (C) 2015 Free Software Foundation, Inc.
  pub  dsa1024/A56E15A3 2002-01-30 Hubert Figuiere <hub@figuiere.net>
  
  Delete this key from the keyring? (y/N) y

  $ gpg2 --keyserver pool.sks-keyservers.net --recv-key A56E15A3
  gpg: key A56E15A3: public key "Hubert Figuiere <hub@figuiere.net>" imported
  gpg: key A56E15A3: "Pedro R. Fernandez (GNU/Linux User #000.204.555

http://counter.li.org/) <prafer@eresmas.com>" not changed

gpg: Total number processed: 2
gpg:               imported: 1
gpg:              unchanged: 1
$ gpg2 --batch --delete-key A56E15A3
gpg: can't do this in batch mode without "--yes"
gpg: (unless you specify the key by fingerprint)
$ gpg2 --yes --batch --delete-key A56E15A3
$ gpg2 -k A56E15A3
pub   dsa1024/A56E15A3 2002-01-30
uid         [ unknown] Hubert Figuiere <hub@figuiere.net>
uid         [ unknown] Hubert Figuiere <hub@mozilla.com>
uid         [ unknown] Hubert Figuiere <hub@abisource.com>
uid         [ unknown] Hubert Figuiere <hfiguiere@teaser.fr>
sub   elg2048/392105DE 2002-01-30
sub   elg2048/DFB0629B 2005-09-21

Thanks for reporting this. This is a bug. BTW, do you have an example of a
short keyid that is ambiguous?

I've tried to improve the web page.

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg-doc.git;a=commit;h=fa61217e26a97c4b9f3294746a581aee5eb47ad8

Since Werner needs to check this, I'm changing the status of this issue to
testing and adding him to the cc.

@Reuben: If you have some ideas of additional improvements, I'd be grateful.
Thanks.

@guilhem, @dkg I've cc'd you on this since you seem to be interested in this code.

I've just updated the branch with a few small bug fixes. Most importantly, I
fixed the memory problem by limiting the read-ahead cache to 20 MB. @guilhem:
I'd be interested to hear whether this fixes the problem that you observed.

@bernhard Sorry for not getting back to you sooner. If you checkout neal/kdb,
you'll get the latest code for the kdb format. Set --homedir or GNUPGHOME
appropriately, import your keyring and then try some operations:

$ mkdir /tmp/gnupg-kbx

$ gpg2 --export | time gpg2 --homedir /tmp/gnupg-kbx --no-default-keyring

--keyring gnupg-kdb:pubring.kdb --import /tmp/keys 2>/dev/null

This makes 6.5 seconds on my box using the kdb format and just under 2 min using
kdb. See
https://lists.gnupg.org/pipermail/gnupg-devel/2015-November/030525.html for some
example benchmarks.

I'm particularly interested to hear whether this fixes the major performance
problem that you're experiencing.

Thanks.

To be clear: the limitation is that GnuPG doesn't currently allow selecting the
main key and subkeys at the same time.

In b64b33b, I've added the ability to update multiple subkeys at once. Note: it
is still not possible to update the main key and the subkeys at the same time,
but this should be a significant improvement, I think.

I've tried to reproduce this issue by setting max-cache-ttl and default-
cache-ttl to 0 in my gpg-agent.conf and running echo | gpg2 -s -a multiple times
in a row. The behavior is such that I have to enter the password each time. I
also traced the control flow in gpg-agent using gdb. The point where the
password is cached is in agent/cache.c:agent_put_cache. The passed ttl is 0 and
opt.def.cache_ttl is also 0. Thus, the password is not cached in my experiments.

Can you please confirm using the most recent version of GnuPG (2.1.9 as of
today) that a gpg-agent.conf with just the above options set caches the password
for you? Does the problem only occur when you are using your gpgme-based
application?

Thanks!

Given that this bug does not appear to be reproducable with 0.9.6, I'm marking
this as resolved.

Ivan: it would be very helpful if you could create a small example demonstrating
the problem. Thanks!

As Werner stated, this appears to be a user support inquiry, which is better
answered elsewhere. As such, I'm marking this issue as resolved.

I'm marking this issue as resolved.

werner: What is your call to action? Should pinentry always be shutdown or is
the status quo acceptable? Thanks.

dkg: I've now applied your backport to the 1.4 branch. Sorry for not doing this
sooner. I believe that this now completely rectifies this issue. As such, I'm
marking this issue as resolve. Thanks.

It seems this now works. In fact the code suggests that it should have detected
this condition since at least 2013 so I don't know what the actual issue was.

$ echo | gpg2 --pinentry-mode=loopback -s -a
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: setting pinentry mode 'loopback' failed: Not supported

Werner notes:

There is a comment in mainproc that we need to sort the list of keys and try
them in an order to get a decryption key early. The other thing is about the
meta data for keys. It would be possible to add a priority to the private keys
and use them to prioritise the list of keys to try.

The error has change in 2.1 to:

  gpg: secret key "foo.gpg" not found: Not found

(i.e., it doesn't say Unknown system error any more.)

The fundamental issue is that the argument to --gen-revoke is not a filename,
but a user id (e.g., the key id). I've accordingly change the error message in
46e128d as follows:

$ gpg2 --output revoke.asc --gen-revoke foo.gpg

gpg: no secret key matches the search term "foo.gpg"

gp_ast: have you gotten a chance to try this?

I'm closing this as Werner think it is a problem with Fedora and the original
reporter hasn't suggested this is not the case.

dkg: No worries. Thanks for testing.

Kristian Fiskerstrand told me that the SKS keyservers currently have a 5 MB
limit for parsing incoming header, pre-merge.

Given how long it has been since this bug was reported, the request for more
information and the lack of response, I'm going to mark this issue as resolved.

It would nice to have an exactly sequence of commands that cause this problem.

This tool has now been marked as deprecated in the documentation.

Based on Werner's comment, this issue has been addressed. As such, I'm closing
this bug report.

I now see I misunderstood the problem description.

The point is that a user has a message that is encrypted to key X. After
receiving the message, he wants to allow another key (say Y) to decrypt the
message by adding a symmetrically encrypted data packet to the message for Y,
i.e., without reencrypting the whole thing.

The reporter wasn't to specify the secret key to use. Werner indicated that
--try-secret-key does what the reporter wants in 2.1, but that this won't be
backported to 2.0. As such, I'm marking this issue as resolved.

I reviewed this issue. I've identified three issues that the reporter is
complaining about:

• Can't create a key with a passphrase (this works)
• Can't import a key that is not protected by a passphrase (this works)
• Can't export a key without protecting it with a passphrase (this is not allowed)

I also moved my mouse between screens in my multi-head setup and gpg did not crash.

I'm marking this issue as resolved.

At least with 2.1.9, it is possible to create a key without a passphrase:

$ gpg2 --gen-key
gpg (GnuPG) 2.1.10-beta132; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
Note: Use "gpg2 --full-gen-key" for a full featured key generation dialog.

GnuPG needs to construct a user ID to identify your key.

Real name: Empty
Email address: empty@testing.org
You selected this USER-ID:

    "Empty <empty@testing.org>"

Change (N)ame, (E)mail, or (O)kay/(Q)uit? o
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
Please enter the passphrase to
protect your new key
Passphrase:
Repeat:
You have not entered a passphrase - this is in general a bad idea!
Please confirm that you do not want to have any protection on your key.

Yes, protection is not needed
Enter new passphrase

[ye]? y

gpg: key BC364B3A marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, TOFU+PGP trust model
gpg: depth: 0 valid: 2 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: next trustdb check due at 2015-11-18
pub rsa2048/BC364B3A 2015-11-18

Key fingerprint = 6766 A52A 3E04 F09B E6F7  F80C 920C 9361 BC36 4B3A

uid [ultimate] Empty <empty@testing.org>
sub rsa2048/906F39F0 2015-11-18

It is also possible to import a secret key that doesn't have a passphrase:

$ gpg --no-options --gen-key
gpg (GnuPG) 1.4.18; Copyright (C) 2014 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:

(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)

Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 1024
Requested keysize is 1024 bits
Please specify how long the key should be valid.

   0 = key does not expire
<n>  = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years

Key is valid for? (0) 10
Key expires at Sat 28 Nov 2015 01:21:43 PM CET
Is this correct? (y/N) y

You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:

    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

Real name: Empty Passphrase
Email address:
Comment:
You selected this USER-ID:

    "Empty Passphrase"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.

You don't want a passphrase - this is probably a *bad* idea!
I will do it anyway. You can change your passphrase at any time,
using this program with the option "--edit-key".

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

Not enough random bytes available. Please do some other work to give
the OS a chance to collect more entropy! (Need 201 more bytes)
...+++++
..+++++
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
.............+++++
.+++++
gpg: unable to use unknown trust model (7) - assuming PGP trust model
gpg: key 4240CFD8 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: public key of ultimately trusted key BC364B3A not found
gpg: public key of ultimately trusted key 41A7057B not found
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 3 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 3u
gpg: next trustdb check due at 2015-11-28
pub 1024R/4240CFD8 2015-11-18 [expires: 2015-11-28]

Key fingerprint = 4E0D 8EED 3567 4228 7F44  C7D7 92BE 30B6 4240 CFD8

uid Empty Passphrase
sub 1024R/D6CF583D 2015-11-18 [expires: 2015-11-28]

$ gpg --no-options --export-secret-key 4240CFD8 > 4240CFD8.sec
$ gpg2 --import 4240CFD8.sec
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: key 4240CFD8: public key "Empty Passphrase" imported
gpg: key 4240CFD8: secret key imported
gpg: Total number processed: 3
gpg: imported: 1
gpg: secret keys read: 3
gpg: secret keys imported: 2
gpg: 3 marginal(s) needed, 1 complete(s) needed, TOFU+PGP trust model
gpg: depth: 0 valid: 3 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 3u
gpg: next trustdb check due at 2015-11-18
$ gpg2 -K 4240CFD8
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
sec rsa1024/4240CFD8 2015-11-18 [expires: 2015-11-28]
uid [ultimate] Empty Passphrase
ssb rsa1024/D6CF583D 2015-11-18 [expires: 2015-11-28]

I'm going to close this. The right forum to address these issues is the OpenPGP
working group.

snicker: I'm sorry it's been so long since you've gotten a response.

Are you still getting these crashes? If so, it would be good if you could run
gpa under gdb. Then, when it crashes, you should be able to get a backtrace
using: 'bt full'

$ gdb ../gpa.exe
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ../build/gnupg/g10/t-keydb...done.
(gdb) run
Starting program: ...

Program received signal SIGABRT, Aborted.
0x00007ffff7149107 in __GI_raise (sig=sig@entry=6)

at ../nptl/sysdeps/unix/sysv/linux/raise.c:56

56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt full
...

Please provide attach the full output to this bug report.

Unfortunately, I don't use Windows so I can't provide more exact instructions.
I hope Andre will correct any details I've gotten wrong.

Given the amount of time since the request for testing, I don't think we are
going to get a response. As such, I'm going to close this issue and mark it as
resolved. If there is still a problem please either reopen this bug report or
file a one. Thanks.

This bug report is 1. actually a support request and 2. lacking information to
properly help the reporter. Although this information was requested, it hasn't
been provided. Given that it has been a year, it is unlikely the user will
provide it now. As such, I'm closing this issue and marking it as resolved.

As I understand the problem, a key appeared in multiple keyrings and this was
causing confusion. I don't think there is a bug here so I'm marking this issue
as resolved.

I'm closing this issue. As Andre already said, a bug report is probably not a
good place to track multiple issues. Further, there hasn't been any activity
for the past year so I'm guessing interest has been lost.

Thanks.

cd2d685 fixes the assert. I don't see the utility of checking keyid (gpg will
do that). Closing.

This has since been corrected. Thanks.

I've removed reference to the issue tracker. The tracker is dead and this page
is only of historic interest. Thanks.

It appears to have been fixed. I'm marking this issue as resolved. Thanks.

The link currently works for me. Either the link was fixed or the target has
been restored. Either way, the issue is now resolved. Thanks.