Some more infos:

https://www.openkeychain.org/faq/#importing-your-own-key-from-gnupg-fails
says that this is a problem for a number of people.

Werner told me that porting the fix back would mean to basically
migrate 2.0 to 2.1, which is useless because 2.1 is already 2.1.

Another possibility would be to change --export to mix public keys (certs)
with secret keys. This would create other problems and thus is not
adviable for a stable version.

So I think this is "won't fix" because it (technically) does not make
sense to fix in 1.4 or 2.0. Solutions: Use 2.1 or wait for 2.2.
As importing implementation: Be tolerant for this problem man use the cert
information if you can.

The keyserver preferences are major privacy problem. They should not be used
and in fact they are ignored in Tor mode. Thus we should not put too much work
in fixing this wish.

6.7 still shows MD5 fingerprints thus switching won't be easy. Does the SHA-256
fingerprint use Base32? If that is the case it might be a serious UX problem
because most people are used to look for colon separated hex digits.

Related issue: #1166.

Now that we have a dirmngr daemon, this should be feasible. I plan to implement
it like this:

Add two flags to the KS_GET command, --enqueue and --drain-queue. --enqueue
merely enqueues the key id and returns immediately, unless --drain-queue is
given.

This will also help us address issue #1827.

After looking at the gnupg 2.0 branch I would say the patch could be applied
to the 2.0 and 2.1 branch to fix the issue in both branches stable/modern
since both version are affected (tested with 2.1.9 and 2.0.29 from gpg2win)

I was looking a bit deeper into the gnupg code and debug messages. As soon as I
plug out the yubikey the usb smartcard reader including the internal smartcard
is no longer available. GnuPG is sending the following messages:

1. this call is still okay 2015-12-06 23:20:31 scdaemon[10940] DBG: enter: apdu_get_status: slot=0

hang=0

2015-12-06 23:20:31 scdaemon[10940] DBG: leave: apdu_get_status => sw=0x0

status=7 changecnt=1

1. here the card is no longer available 2015-12-06 23:20:31 scdaemon[10940] DBG: enter: apdu_get_status: slot=0

hang=0

2015-12-06 23:20:31 scdaemon[10940] pcsc_get_status_change failed: no

service (0x8010001d)

2015-12-06 23:20:31 scdaemon[10940] DBG: leave: apdu_get_status =>

sw=0x1000b status=7 changecnt=1

Error Message 0x8010001d == SCARD_E_NO_SERVICE (The Smart card resource manager
is not running.)
https://pcsclite.alioth.debian.org/api/group__ErrorCodes.html#gad4729ab109ff490
285d2ad881c04bee8

Now there's an internal mapping happing from 0x8010001d to sw=0x1000b
http://git.gnupg.org/cgi-bin/gitweb.cgi?
p=gnupg.git;a=blob;f=scd/apdu.c;h=95a25611b7ff46c87e2e888643bec0a10454f894;hb=H
EAD#l899

The SCD internal resulting error is 0x1000b means SW_HOST_GENERAL_ERROR which
is the default error if nothing else has matched yet.

The next lines in the logfiles are showing a different pcsc error code.

2015-12-06 23:20:32 scdaemon[10940] DBG: enter: apdu_get_status: slot=0

hang=0

2015-12-06 23:20:32 scdaemon[10940] pcsc_get_status_change failed: service

stopped (0x8010001e)

2015-12-06 23:20:32 scdaemon[10940] DBG: leave: apdu_get_status =>

sw=0x1000b status=7 changecnt=1

Error Message 0x8010001d == SCARD_E_SERVICE_STOPPED (The Smart card resource
manager has shut down.)
https://pcsclite.alioth.debian.org/api/group__ErrorCodes.html#ga262c34297ab1b65
db1c9516ccc0dd9a0

The pcsc error code is still mapped to the generic error code (0x1000b)

--

possible resolution: I've no clue if the error codes 0x8010001d or 0x8010001e
are thrown in different scenarios. But if we would map those 2 messages to the
internal SW_HOST_NO_READER error the scdaemon would like remove the (likely
disconnected usb) reader from the current list, resulting in freeing up the
used card.

I can't verify this idea cause I'm not able to compile gnupg under windows but
I've attached a solution patch in this ticket

D339: 732_0001-scd-Fix-removal-of-unplugged-usb-readers.patch

I generalized the ssh key fingerprinting code so that we can select the digest algorithm.

Now I'm a little unsure how to proceed. We can easily include both the MD5 and the SHA256 digest
in the sshcontrol file. But what shall we use for expanding '%F' in key descriptions? If we
transition too soon or too late, users might not recognize their key. Displaying both surely is
too verbose. We could make it configurable, or at least a compile time option.

What do you think?

Err, fixed in 6ac57a48.

Fixed in

Fixed in a8308ba5.

% g10/gpg2 --keyserver hkp://keyring.debian.org --search-keys dkg
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: error searching keyserver: Not implemented
gpg: keyserver search failed: Not implemented

Well ... the udev can only be a workaround. Killing the scdaemon from the
outside is not the correct way to handle this issue.

scdaemon should realize it itself that it's connection to the smartcard has
been stopped/killed/canceled/disconnected (whatever) and kill/reset and
restart it's internal process leading to removing the previously loaded card
from the memory.

There should be no need for an outside trigger.

I tested this with a fresh windows10 (which has never seen a yubikey
before). I installed gnupg 2.1.9 binaries and gnupg was able to speak with
the card with no additional drivers needed. (the problem with disconnecting
the card still occours).

And without the need of special drivers there should be no need for any
special rules to be applied.

The problem here is that the hkp client code folds all http status codes other
than 200 and 3xx into GPG_ERR_NO_DATA. This is also a problem for issue #1038.

In the meantime, a workaround is to add an udev rule to scdaemon whenever the
Yubikey is plugged in:
https://github.com/Yubico/yubikey-personalization/issues/36 . I guess it doesn't
work on Windows, though.

OK, so things have changed regarding how this is handled since 2.1. That’s
probably why people like me think it’s still bogus, because behind the true bug
there was also another underlying change.

I can confirm it now works once correctly configured. Thanks for your help.

This, 2123 and 2130 can be closed I think.

On Wed, Dec 02, 2015 at 12:55:23PM +0000, Justus Winter via BTS wrote:

Justus Winter <justus@g10code.com> added the comment:

I can reproduce this without the proper configuration described in https://sks-
keyservers.net/overview-of-pools.php#pool_hkps:

I'm not sure, I reverted said change, and it still works for me:

% echo -e "KEYSERVER hkp://ipv6.pool.sks-keyservers.net/\nKS_SEARCH CADE3658\n"

dirmngr[10105.0]: marking host '[2a01:4f8:192:f5::3]' as dead
dirmngr[10105.0]: marking host '[2001:41d0:2:a8b4::10]' as dead
dirmngr[10105.0]: marking host '[2001:67c:2050:1000::3:4]' as dead
dirmngr[10105.0]: marking host 'hufu.ki.iif.hu' as dead

The log clearly states the problem:

2015-10-09 10:27:37 dirmngr[2516.0] TLS verification of peer failed: The
certificate is NOT trusted. The certificate issuer is unknown.

Please see https://sks-keyservers.net/overview-of-pools.php#pool_hkps for how to
configure gpg properly. With the CA for the pool, this works as expected.
(remember to kill the old dirmngr daemon):

% gpg2 --keyserver hkps://hkps.pool.sks-keyservers.net --recv-keys
5EE1DBA789C809CB
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: key 89C809CB: public key "git-annex distribution signing key (for Joey
Hess) <id@joeyh.name>" imported
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: Total number processed: 1
gpg: imported: 1

I can reproduce this without the proper configuration described in https://sks-
keyservers.net/overview-of-pools.php#pool_hkps:

% :> /home/teythoon/repos/g10/local/gnupghome/dirmngr.conf
% gpg2 --keyserver hkps://hkps.pool.sks-keyservers.net --search-keys 2071B08A33BD3F06
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: error searching keyserver: General error
gpg: keyserver search failed: General error

But with it, it seems to work fine. Remember to kill the old daemon first:

% echo hkp-cacert /home/teythoon/repos/g10/sks-keyservers.netCA.pem >
/home/teythoon/repos/g10/local/gnupghome/dirmngr.conf
% pkill dirmngr
% gpg2 --keyserver hkps://hkps.pool.sks-keyservers.net --search-keys 2071B08A33BD3F06
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: data source: https://jarvis.alpha-labs.net:443
(1) NIIBE Yutaka (GnuPG Release Key) <gniibe@fsij.org>

          2048 bit RSA key 33BD3F06, created: 2014-10-29, expires: 2016-10-28

You can talk to the dirmngr directly like this:

% echo -e "KEYSERVER hkps://hkps.pool.sks-keyservers.net\nKS_SEARCH 2071B08A33BD3F06\n" | dirmngr

If this still does not work for you, please paste the output of the above invocation.

'gpg-zip' is being phased out. It will be shipped with gpg-classic, and dropped
from gpg-modern. Furthermore, I just checked that we no longer install gpg-
zip.1.

UPDATE: The latest gpg on an AIX box that I could get my hands on does not work
at all, so until that problem is resolved I cannot really test further.

When we invoke gpgme a process just hangs, but as far as we can tell from the
strace, gpg is busy closing file descriptors in the entire file descriptor
range! If we limit the file descriptor range to a smaller number (by, say,
"limit descriptors 4096 ; limit -h descriptors 4096"), we get a fast failure.

The gpg versions we have installed are 2.0.26 and 1.4.18. It seems that the
pinentry program never even gets invoked. Attached is a program you can test with.

Right, or for example to re-encrypt a message to a workmate.

I have no examples. However I have seen too many scripts using gpg in often
very strange ways. Thus I always hesitate to change anything even if it is
clearly stated that this is not a documented interface.

People often hack something, it works for them, and they continue with other
work. Years later things break and nobody is left knowing what the code
actually does. Not good but that is how things are.

There is no guarantee how long the output will be. Several factors influence
this. For example the compression or removed leading zero bytes in encrypted
random session keys can have this effect.

If you have more questions about this, please direct them to one of the mailing
lists.

I just double checked the code from 2.1 - it looks really okay.
I need to look at the 2.x branch, though.

FWIW, with commit 19545e3a from 2015-09-09 I had bumped the limit up to 20MiB.
This should solve all current practical problems.

Test data from: http://keyserver.borgnet.us/dump/sks-dump-0000.pgp.bz2

In one console window:
mkdir c:\test-issue2135
set GNUPGHOME=c:\test-issue2135
gpg2 --import c:\users\aheinecke\Desktop\sks-dump-0000.pgp

in another:
set GNUPGHOME=c:\test-issue2135
gpg2 -k

Triggers this: (And the error messages also look wrong)

gpg: waiting for lock c:/test-issue2135/pubring.gpg.lock...
gpg: renaming c:/test-issue2135/pubring.gpg' to c:/test-issue2135/pubring.bak'
failed: Permission
denied
gpg: error writing keyring `c:/test-issue2135/pubring.gpg': Permission denied
gpg: key CBB511F4: public key "[User ID not found]" imported
gpg: error reading `c:\\Users\\aheinecke\\Desktop\\sks-dump-0000.pgp':
Permission denied
gpg: import from `c:\\Users\\aheinecke\\Desktop\\sks-dump-0000.pgp' failed:
Permission denied
gpg: Total number processed: 278
gpg: w/o user IDs: 14
gpg: imported: 265 (RSA: 82)
gpg: renaming c:/test-issue2135/pubring.gpg' to c:/test-issue2135/pubring.bak'
failed: Permission
denied
gpg: failed to rebuild keyring cache: Permission denied
gpg: no ultimately trusted keys found

werner: Are you suggesting that there are programs that use an ambiguous
specification in order to delete just one key? I'm trying to imagine how this
is not undesirable behavior. Do you have any examples of this?

neal: What you describe are two a different things/bugs.

1. Indicating that a specification is ambigious. We do this at other places as well

2. Continue to show (with -k) or ask (with --delete-key) the other matching keys.

For the later I don't think that there is a need to do this with --delete-key
because we would have already printed that the specification is ambigious.
However a "gpg -k shortkeyid" should list all matching short keyids. A quick
test with:

0F26563A76B8337A
D1EC50AA76B8337A

shows that this is not the case. Note that we need to take care when modifiying
--delete-key. Not all programs using this are well written but implicity assume
the current behavious. And unfortunately many authors do not use --with-colons
and rely on the human output. Not deleting is better than accidently deleting
something.

In this case I'm pretty sure that it does not. I check that I can come up with a
testcase that does not involve kleo.

Is Kleopatra messing around with files in ~/.gnupg in any way? IIRC, Kleo
sometimes bypasses gpgme. For example does it open pubring.gpg ?

No, keys were never moved. I have a test case ready, but I am trying to get a
hold of a newer AIX box to test it there. I will upload it shortly.

Ivan: Did you create the key on a fast machine and then moved it to a slower
one. Each use of a passphrases takes about 100ms for security reasons. Use
"gpg --passwd KEYID" to change the passphrase and adjust it to the speed of your
system.

A correctly installed system will start the agent either on the fly only the
first time or you need to start gpg-agent manually. See the man page for details.

May this be related to you fix for

if(ai->family != AF_INET6)

commit 23ea641b ?

Please run

  gpg --with-keygrip --with-fingerprint --with-fingerprint -K 30A99F9A

and

  gpg --with-keygrip --with-fingerprint --with-fingerprint -K 9BA84708

If one of the commands does not show a key run it again with -k
(lowercase). Also run

gpg --version

OK, based on my findings, I have to clarify that it is the environment of the
agent program that is cached, thus rendering the pinentry program oblivious to
the further changes of the environment, even though it is those changes that
"guide" it in the retrieval of the correct passphrase.

Attached is a test program. I apologize that it is TCSH, but that is what we use
in our shop (for portability). Running test.csh should produce an output like
this:

    > test.csh < /dev/null
    ---------------------------------------------------
     GPG version:
    ---------------------------------------------------
    gpg (GnuPG) 2.1.9
    libgcrypt 1.6.3
    Copyright (C) 2015 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    
    Home: /tmp/gpg-caching/.gnupg
    Supported algorithms:
    Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
    Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
            CAMELLIA128, CAMELLIA192, CAMELLIA256
    Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
    Compression: Uncompressed, ZIP, ZLIB, BZIP2
    
    ---------------------------------------------------
     Available keys:
    ---------------------------------------------------
    /tmp/gpg-caching/.gnupg/pubring.kbx
    -----------------------------------
    pub   dsa1024/E65831AC 2015-11-24
    uid         [ultimate] user1 <user1@email.com>
    sub   rsa2048/95F85202 2015-11-24
    
    pub   dsa1024/B0731F16 2015-11-24
    uid         [ultimate] user2 <user2@email.com>
    sub   rsa2048/67841B64 2015-11-24
    
    
    ---------------------------------------------------
     Key 1 decrypted using gpg2 directly:
    ---------------------------------------------------
    gpg: encrypted with 2048-bit RSA key, ID 95F85202, created 2015-11-24
          "user1 <user1@email.com>"
    32-bit-long key for user1 (ONE).
    
    ---------------------------------------------------
     Key 2 decrypted using gpg2 directly:
    ---------------------------------------------------
    gpg: encrypted with 2048-bit RSA key, ID 67841B64, created 2015-11-24
          "user2 <user2@email.com>"
    gpg: public key decryption failed: Broken pipe  
    gpg: decryption failed: No secret key
    
    
    ---------------------------------------------------
     Key 1 decrypted using gpgme:
    ---------------------------------------------------
    32-bit-long key for user1 (ONE).
    
    ---------------------------------------------------
     Key 2 decrypted using gpgme:
    ---------------------------------------------------
    Error while accessing key file user2.key: Decryption failed

This test creates two private/public key pairs in the same keyring; the password
to the private key of the first is "password1"; that of the second is
"password2". It then generates two files (user1.key and user2.key), one encrypted
with the first public key, the other with the second. The contents of those files
are "32-bit-long key for user1 (ONE)." and "32-bit-long key for user2 (TWO).",
respectively. The test also involves a simplistic pinentry program that returns
the value of $password to the agent. The point is to demonstrate that once the
gpg agent is started, it caches the environment, such that even updating
$password to the right value is of no use. And that is why only user1.key is
decrypted successfully (the agent was started to decrypt it).

Very conveniently in git:

Found using the Clang Static Analyzer.
Signed-off-by: Justus Winter <justus@g10code.com>

Please describe problems and do not just send a patch. You may however send a
patch to gnupg-devel.

Fixed with commit eb957ffc. Thanks.

However, this is not a problem because that value is not used.

If you check the do-while above you will notice that after the loop LINE is
guaranteed to always end with a LF. Thus strpbrk will always succeed.

Werner, in https://lists.gnupg.org/pipermail/gnupg-users/2015-May/053617.html you wrote:

The real bug is that dirmngr does not mark the v6 address dead and
retry anotyer server (or the v4 address).

I cannot reproduce this. I pointed dirnmngr to ipv6.pool.sks-keyservers.net and servers
got marked as dead as expected.

@guilhem, @dkg I've cc'd you on this since you seem to be interested in this code.

I've just updated the branch with a few small bug fixes. Most importantly, I
fixed the memory problem by limiting the read-ahead cache to 20 MB. @guilhem:
I'd be interested to hear whether this fixes the problem that you observed.

@bernhard Sorry for not getting back to you sooner. If you checkout neal/kdb,
you'll get the latest code for the kdb format. Set --homedir or GNUPGHOME
appropriately, import your keyring and then try some operations:

$ mkdir /tmp/gnupg-kbx

$ gpg2 --export | time gpg2 --homedir /tmp/gnupg-kbx --no-default-keyring

--keyring gnupg-kdb:pubring.kdb --import /tmp/keys 2>/dev/null

This makes 6.5 seconds on my box using the kdb format and just under 2 min using
kdb. See
https://lists.gnupg.org/pipermail/gnupg-devel/2015-November/030525.html for some
example benchmarks.

I'm particularly interested to hear whether this fixes the major performance
problem that you're experiencing.

Thanks.

Fixed in a9e0b1dd.

To be clear: the limitation is that GnuPG doesn't currently allow selecting the
main key and subkeys at the same time.

In b64b33b, I've added the ability to update multiple subkeys at once. Note: it
is still not possible to update the main key and the subkeys at the same time,
but this should be a significant improvement, I think.

I've tried to reproduce this issue by setting max-cache-ttl and default-
cache-ttl to 0 in my gpg-agent.conf and running echo | gpg2 -s -a multiple times
in a row. The behavior is such that I have to enter the password each time. I
also traced the control flow in gpg-agent using gdb. The point where the
password is cached is in agent/cache.c:agent_put_cache. The passed ttl is 0 and
opt.def.cache_ttl is also 0. Thus, the password is not cached in my experiments.

Can you please confirm using the most recent version of GnuPG (2.1.9 as of
today) that a gpg-agent.conf with just the above options set caches the password
for you? Does the problem only occur when you are using your gpgme-based
application?

Thanks!

Thank you for quick response)

I couldn't build and test 2.1.9 right now, but bug is still here in 1.4.16 and
2.0.22.

I've created test script for this case:
https://gist.github.com/anton-ryzhov/a0dcfcaabe18fc6ad35e

Run ./gen.sh in some working folder and then try ./runtest.sh several times,
expire different subkeys, compare the result.

Ivan: it would be very helpful if you could create a small example demonstrating
the problem. Thanks!

Keep the bug open. We won't fix it for the next release.

As Werner stated, this appears to be a user support inquiry, which is better
answered elsewhere. As such, I'm marking this issue as resolved.

I'm marking this issue as resolved.