Page MenuHome GnuPG

gpg2 prefers missing secret key to available key on card
Closed, ResolvedPublic


This is my keyring structure:


sec rsa2048/28883ECB 2013-09-29 [SC]
uid [ultimate] James M Wilson <jmw@xxxxxx>
uid [ultimate] James M Wilson <jmw@xxxxxx>
ssb> rsa2048/8A41DB6A 2013-09-29 [E]
ssb> rsa2048/0E92F55D 2014-11-09 [S] [expires: 2016-11-08]
ssb> rsa2048/6965D0C5 2014-11-09 [A] [expires: 2016-11-08]
ssb# rsa2048/793D4BD1 2014-11-09 [S] [expires: 2016-11-08]
ssb# rsa2048/A3A940E5 2014-11-09 [A] [expires: 2016-11-08]

Three subkeys for E,S,A purposes are on a card. Two S and A subkeys are missing
since they are on a different card but I have the public keys.

Unless I use "-u 0E92F55D!" or set default-key to the same in gpg.conf, I cannot
sign things.

gpg2 -b test
gpg: signing failed: No secret key
gpg: signing failed: No secret key

0E92F55D is the only usable signing subkey. In this situation shouldn't gpg
ignore the "ssb#" key and use the one that is available?



Event Timeline

jwilson set Version to 2.1.4.
jwilson added a subscriber: jwilson.

No, I don't think so. You created a newer subkey on a smartcard and thus gpg
assumes that you want to use that key.

It should actually ask you to insert that card - doesn't it do that? There is
an open bug which might prevent that gpg-agent asks for the correct card - is
that the case? The missing "Card serial no. = nnnn kkkkkkkk" may indicate that.

Did you ever used the cards with that version of gpg and did you run a "gpg

I removed the stub keys for the last two, that is why they are listed as "ssb#"
instead of "ssb>".

If the expected behavior is newest key is always preferred, than that's fine and
easy to work around with default-key, although it would be nice to exclude
unusable keys.

Yes, the newest subkey will be preferred. This is more an doc or FAQ issue.

werner renamed this task from gpg2 prefers missing secret key to available key to faq: gpg2 prefers missing secret key to available key.Oct 28 2015, 7:13 PM
werner closed this task as Invalid.
werner added a project: FAQ.
werner removed a project: Bug Report.

I don't think this is a doc or FAQ issue, i think it's an actual bug that has a
significant effect on usability.

If gpg has an available key that would work, it should use it, rather than
preferring the unavailable key.

If the user explicitly specifies an unavailable subkey then sure, gpg should
fail. But if they've only specified their primary (or their UID) then gpg
should be willing to use any available active (non-revoked, non-expired) subkey
with the right usage flags instead of failing if an unavailable one has a newer

I'm changing this from "nobug" to "bug", because it is clearly causing problems
for people with separate per-device signing keys, or with multiple smartcards
(e.g. work and home)

dkg added a project: Bug Report.

I personally am affected by this as well in a couple cases.

This is the topology of my keys:
sec# rsa4096/0x703E78EA22A5ABAB 2015-12-30 [SC] [expires: 2016-12-29]
uid [ultimate] JD Friedrikson (Personal Mail Server)
uid [ultimate] JD Friedrikson (Gmail Address)
uid [ultimate] JD Friedrikson (Linode Address)
uid [ultimate] [jpeg image of size 5874]
ssb rsa4096/0x60E6AFFEEC378639 2015-12-30 [E] [expires: 2016-12-29]
ssb rsa4096/0xC6C7A50DF6FC94C4 2015-12-30 [S] [expires: 2016-12-29]
ssb# rsa4096/0xC5197712F5411047 2015-12-30 [S] [expires: 2016-12-29]
ssb# rsa4096/0x4989B27BD7E45F52 2015-12-30 [S] [expires: 2016-12-29]
ssb# rsa4096/0x04B3529A021FB930 2015-12-30 [S] [expires: 2016-12-29]

I have detached signing subkeys for each device. While I do understand that I
can explicitly force subkey selection with "-u <subkeyid>!" on the commandline
with gpg2, I do not have the option when using programs that are either built as
a front-end for gpg2 (enigmail) or implement gpg in some way (git).

For example, when I try signing a commit with git this is what I get:
λ ~/test/ master* git config --global user.signingkey "0xC6C7A50DF6FC94C4"
λ ~/test/ master* git commit -a -S -m "test"
gpg: signing failed: No secret key
gpg: signing failed: No secret key
error: gpg failed to sign the data
fatal: failed to write commit object

Alright, sure we can try adding the "!" to see if we can force it:
λ ~/test/ master* git config --global user.signingkey '0xC6C7A50DF6FC94C4!'
λ ~/test/ master* git commit -a -S -m "test"
gpg: signing failed: Inappropriate ioctl for device
gpg: signing failed: Inappropriate ioctl for device
error: gpg failed to sign the data
fatal: failed to write commit object

I'm relatively sure that git is having trouble parsing the attempt to force the

And if anything else, it does not make sense to me why the default behavior
would be to reach for subkeys that aren't even in the private keyring. I get
that it's going for the newest subkey first, but maybe the behavior should be
newest AND present instead.

To add, why not also enable forcing a certain subkey without use of the "!"? I
figure that the only reason it's written that way would be in compliance with
the default behavior.

That way, you could make git work better with different subkeys if you wanted to
use a separate subkey dedicated to only signing git commits and tags. Both the
current and the behavior of "newest AND present" wouldn't help if you wanted to
do that, but if you could force a subkey without the "!" then you could easily
have more flexibility in choosing subkeys for git.

In reference to that last part about having a dedicated subkey for git, I
realized that I should probably just make a separate master key. Please ignore that.

dkg raised the priority of this task from Normal to High.Apr 19 2017, 5:46 PM
dkg updated the task description. (Show Details)
gniibe added a subscriber: gniibe.

We need to change how to access scdaemon from gpg frontend. Thus, I merge this to T2291: Smartcard interaction improvement (was: Shadowed private key design (for smartcard)), setting its priority to High. Please continue at T2291.

Sorry, merging/closing is not good. This should be a subtask of T2291.

For signing (not sign to key), here is my attempt:

I'm not yet sure about other impact.

I remove FAQ tag, because this is not general problem for preference on new subkey, but a suggestion for preference on available key on card.

gniibe renamed this task from faq: gpg2 prefers missing secret key to available key to gpg2 prefers missing secret key to available key on card.Apr 26 2017, 4:51 AM

Sorry, I was wrong. The patch also works for signing to key.
The impact is gpg frontend always asks gpg-agent for card key.
It involves invoking scdaemon and accessing to USB.

I pushed the change to the repository.

The interaction is not yet perfect. There are cases: When the card key is not available, gpg frontend still tries to access the card key.

Sorry, my fix was not good. Re-opening.

> gpg --version

gpg (GnuPG) 2.1.21
libgcrypt 1.7.8

Note that gpg --card-status fails to create secret key stubs when no real secret key is present on a computer.
This is the case for example when you connect the smart card to a new computer and fetch your public key:

> gpg --homedir /tmp/.gnupg --card-edit
gpg/card> fetch                                                                                                                                                                                                                               
gpg: requesting key from '...someurl'                                                                             
gpg: /tmp/.gnupg/trustdb.gpg: trustdb created                                                                                                                                                                                                 
gpg: key A3AA60EA350A63A4: public key "userid" imported                                                                                                                                                             
gpg: Total number processed: 1                                                                                                                                                                                                                
gpg:               imported: 1

gpg/card> quit

> gpg --homedir /tmp/.gnupg --card-status --debug-all

gpg: DBG: chan_3 -> SCD GETATTR KEY-ATTR                   
gpg: DBG: chan_3 <- S KEY-ATTR 1 1 rsa2048 32 1            
gpg: DBG: chan_3 <- S KEY-ATTR 2 1 rsa2048 32 1                       
gpg: DBG: chan_3 <- S KEY-ATTR 3 1 rsa2048 32 1            
gpg: DBG: chan_3 <- OK                                                
gpg: DBG: [not enabled in the source] keydb_new            
gpg: DBG: [not enabled in the source] keydb_search enter              
gpg: DBG: keydb_search: 1 search descriptions:             
gpg: DBG: keydb_search   0: AAA20: 'A176 8EAE 3A4C ACF2 2566  6A3B 2CD9 4A7A 642A EAE2'                                
gpg: DBG: keydb_search: searching keybox (resource 0 of 1)     
gpg: DBG: keydb_search: searched keybox (resource 0 of 1) => EOF                                                       
gpg: DBG: [not enabled in the source] keydb_search leave (not found)                                                   
gpg: DBG: [not enabled in the source] stop                               

Signature key ....: signaturekeyid
      created ....: date
Encryption key....: encryptionkeyid
      created ....: date
Authentication key: authenticationkeyid
      created ....: date
General key info..: [none]

As you can see above gpg --card-status fails to find the key and does not create stubs.
Currently, the workaround for this issue is to import the secret keys from backup and let gpg replace them with stubs.
Please, fix this!
Thank you!

@fogine , I'm afraid your comment is related to this bug particular report of T1983: gpg2 prefers missing secret key to available key on card.
And your problem cannot be replicated by my environment with 2.1.22.
If you still have the issue with 2.1.22, please open new ticket.

gniibe removed a project: Restricted Project.

Now, 2.1.22 or later supports automatic selection of secret key by available key on card.

I think that this ticket and handle different things, although both do key selection.

This old bug was that: despite no on-disk secret key, card key was not selected. It has been fixed.

Recently, for the change of T3416: gpg should select available signing key on card (even with -u option), it was further improved. Now, key selection is done with the information of on-line card keys' availability.

But, in a case like, still, user have to specify the subkey on disk specifically. If not specified, gpg asks the user to insert a card. (It is possible a user has the card in his pocket.)

Well, when the on-disk key is cached, it is preferred than the key on card which is not on-line. So, now, there is a workaround for 346241:
Do the following beforehand,

$ gpg-connect-agent "PRESET_PASSPHRASE <KEYGRIP>"

Then, the on-disk key will be used.