This has been fixed in libgpg-error and the gpg-error.m4 macros have been
updated in all gnupg related libraries.

This has already been fixed in the repo:
commit c01c8f0c4f55d76b037c7f6aa44ad25ede18d38a
1.19 will soon be released.

This will mostly be fixed in 2.1.3 however one bug still persists: You need to
enter the emprty passphrase twice. This annoys me too and it will be fixed
after 2.1.3.

Note: for more information about this issue, please refer to:

  T1945

  https://wiki.gnupg.org/GnomeKeyring

(I've added this here, since this page is one of the top hits on ddg and google
when searching for the warning message.)

Here's the lsusb output:

Bus 001 Device 002: ID 058f:9540 Alcor Micro Corp.
Device Descriptor:

bLength                18
bDescriptorType         1
bcdUSB               2.01
bDeviceClass            0 (Defined at Interface level)
bDeviceSubClass         0 
bDeviceProtocol         0 
bMaxPacketSize0         8
idVendor           0x058f Alcor Micro Corp.
idProduct          0x9540 
bcdDevice            1.20
iManufacturer           1 Generic
iProduct                2 EMV Smartcard Reader
iSerial                 0 
bNumConfigurations      1
Configuration Descriptor:
  bLength                 9
  bDescriptorType         2
  wTotalLength           93
  bNumInterfaces          1
  bConfigurationValue     1
  iConfiguration          0 
  bmAttributes         0xa0
    (Bus Powered)
    Remote Wakeup
  MaxPower               50mA
  Interface Descriptor:
    bLength                 9
    bDescriptorType         4
    bInterfaceNumber        0
    bAlternateSetting       0
    bNumEndpoints           3
    bInterfaceClass        11 Chip/SmartCard
    bInterfaceSubClass      0 
    bInterfaceProtocol      0 
    iInterface              0 
    ChipCard Interface Descriptor:
      bLength                54
      bDescriptorType        33
      bcdCCID              1.10  (Warning: Only accurate for version 1.0)
      nMaxSlotIndex           0
      bVoltageSupport         7  5.0V 3.0V 1.8V 
      dwProtocols             3  T=0 T=1
      dwDefaultClock       3700
      dwMaxiumumClock     12000
      bNumClockSupported      3
      dwDataRate           9946 bps
      dwMaxDataRate      688172 bps
      bNumDataRatesSupp.    138
      dwMaxIFSD             254
      dwSyncProtocols  00000007  2-wire 3-wire I2C
      dwMechanical     00000000 
      dwFeatures       000404BE
        Auto configuration based on ATR
        Auto activation on insert
        Auto voltage selection
        Auto clock change
        Auto baud rate change
        Auto PPS made by CCID
        Auto IFSD exchange
        Short and extended APDU level exchange
      dwMaxCCIDMsgLen       272
      bClassGetResponse    echo
      bClassEnvelope       echo
      wlcdLayout           none
      bPINSupport             0 
      bMaxCCIDBusySlots       1
    Endpoint Descriptor:
      bLength                 7
      bDescriptorType         5
      bEndpointAddress     0x81  EP 1 IN
      bmAttributes            3
        Transfer Type            Interrupt
        Synch Type               None
        Usage Type               Data
      wMaxPacketSize     0x0004  1x 4 bytes
      bInterval               1
    Endpoint Descriptor:
      bLength                 7
      bDescriptorType         5
      bEndpointAddress     0x02  EP 2 OUT
      bmAttributes            2
        Transfer Type            Bulk
        Synch Type               None
        Usage Type               Data
      wMaxPacketSize     0x0010  1x 16 bytes
      bInterval               0
    Endpoint Descriptor:
      bLength                 7
      bDescriptorType         5
      bEndpointAddress     0x83  EP 3 IN
      bmAttributes            2
        Transfer Type            Bulk
        Synch Type               None
        Usage Type               Data
      wMaxPacketSize     0x0010  1x 16 bytes
      bInterval               0

Binary Object Store Descriptor:

bLength                 5
bDescriptorType        15
wTotalLength           12
bNumDeviceCaps          1
USB 2.0 Extension Device Capability:
  bLength                 7
  bDescriptorType        16
  bDevCapabilityType      2
  bmAttributes   0x00000002
    Link Power Management (LPM) Supported

Device Status: 0x0000

  (Bus Powered)

For the scdaemon log, do you need it:

• with pcscd running or with GnuPG direct ccid implementation?
• in “working” condition (for example doing a gpg --card-status or gpg --sign)?
• during the “breakage” (doing a gpg --decrypt)
• in “broken” condition (after doing a gpg --decrypt).

Sorry if my report wasn't so clear. The broken behavior only appears:

• when using GnuPG ccid implementation (instead of pcscd);
• when doing a decrypt operation (maybe also an encrypt, I didn't check yet, but I'd be surprised since the smartcard hardly do any job here)

After trying a decrypt operation, the USB reader is in a non working condition, and I can only restore working condition by doing a reboot (I'v
tried to cut power to the USB bus but that doesn't seem enough).

Let me confirm. Does this bus still exist in recent version of gpg 1.4 and/or
2.0, 2.1?

Please give me the output of lsusb -v -d 058f:9540
and debug log of scdaemon.
Do you mean --card-status works bug --decrypt fails?

The attached callgraph for g10/gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID4
(without the patch) shows that a significant amount of time is spent
fread(3)'ing each blob of the keybox (for every single sig encountered).

A caching mechanism (ideally two hashtables, the first mapping 64-bits key IDs
to their primary UID, the second keeping track of unknown signer's 64-bits key
IDs) would certainly speed up things, but as shown in T1938 (guilhem on Apr 01 2015, 04:12 PM / Roundup) read(2) causes
overhead on a keybox for *both* --list-keys and --list-sigs, so it wouldn't
solve the core of the issue. I would therefore agree with dkg's T1938 (dkg on Apr 01 2015, 09:40 PM / Roundup) here:
it's probably worth trying to mmap(3) the keybox instead of fread(3)'ing it.

Aren't the second tests (gpg2 --homedir /tmp/gnupg1 --list-sigs) precisely
what you want? This suggests that the bug is triggered by get_user_id, but
only when used on a keybox.

Using master (6619ead) without your patch yields timings comparable to 2.1.2
with both pubring.kbx and pubring.gpg. As I hinted at in the ExtLink, I also
noticed that adding --fast-list-mode makes --list-sigs as fast as --list-keys
(which is not surprising, since it removes the quadratic behavior). Since it
also doesn't show any significant difference between pubring.gpg and
pubring.kbx, it confirms that the bug is triggered by get_user_id when used on
a keybox (for some reason read(2) seems to be much much more expensive on a
keybox than on the legacy format).

Not yet.

I have no immediate idea why 1.4 is faster than 2.1. To sort out whether it is
a problem of the gnupg version or of the keyring format you may replace
pubring.kbx in gnupg2 by pubring.gpg from gnupg1 and check whether there is a
substantial difference between the evrsions using the same pubring.gpg.

Given the number of keys the default cache sizes should work.

I am not testing with 2.1.2 but with master which might have some updates. With
-fast-list-mode --lits-sigs is really fast.

Taking the primary user id from the keybox meta data is not a good idea because
it violates the layering.

For a regular private key wie have such an indentifier. We don't have it for
symmetric passphrases but they are very rarely used. There is also no need to
have any cache for a smart card PIN.

The OpenPGP information as conveyed with SETDESC ist not a stable idnetification
but I think I can add something else. Not for 2.1.3 but soon after it.

Indeed, your patch speeds up the listing, but doesn't seem to fully solve the
regression and in particular doesn't make the keybox setting as fast as the
legacy pubring.gpg one.

Here is my test environement:

• fresh identical keyrings /tmp/gnupg1 and /tmp/gnupg2, the former with a

pubring.gpg, the latter with a pubring.kbx.

• 1246 public keys: `gpg2 --homedir /tmp/gnupg1 --with-colons --list-sigs |

grep -c '^pub:'`

• 228176 sigs: `gpg2 --homedir /tmp/gnupg1 --with-colons --list-sigs | grep -Ec

'^(sig|rev):'`

• 156625 sigs from an unknown signer: `gpg2 --homedir /tmp/gnupg1 --with-colons

--list-sigs | grep -Ec '^(sig|rev):([^:]*:){8}\[User ID not found\]:'`

• 13754 unknown signers: `gpg2 --homedir /tmp/gnupg1 --with-colons --list-sigs

found\]:.*/\3/p' | sort -u | wc -l`

• g10/gpg2 is the current HEAD (6619ead) with your patch on top; gpg2 is

2.1.2; gpg is 1.4.18.

• $keyID1: 216 sigs (98 unique signers), 178 from an unknown signer (87 unique

unknown signers)

`gpg --homedir /tmp/gnupg1 --list-sigs $keyID1`:      0:07.17 (5.24 user,

1.82 sys) 7956k maxres

`gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID1`:     0:07.62 (5.47 user,

1.53 sys) 8312k maxres

`gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID1`:     0:58.27 (3.74 user,

51.69 sys) 11836k maxres

`g10/gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID1`: 0:06.71 (5.03 user,

1.58 sys) 8224k maxres

`g10/gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID1`: 0:17.03 (1.50 user,

14.92 sys) 11812k maxres

• $keyID2: 1026 sigs (250 unique signers), 716 from an unknown signer (181

unique unknown signers)

`gpg --homedir /tmp/gnupg1 --list-sigs $keyID2`:      0:07.29 (5.33 user,

1.86 sys) 7940k maxres

`gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID2`:     0:07.53 (5.68 user,

1.64 sys) 8256k maxres

`gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID2`:     1:01.09 (3.59 user,

53.18 sys) 11784k maxres

`g10/gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID2`: 0:06.66 (4.98 user,

1.55 sys) 8052k maxres

`g10/gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID2`: 0:17.11 (1.50 user,

15.16 sys) 11812k maxres

• $keyID3: 1334 sigs (302 unique signers), 374 from an unknown signer (80 unique

unknown signers)

`gpg --homedir /tmp/gnupg1 --list-sigs $keyID3`:      0:18.31 (12.18 user,

5.71 sys) 8928k maxres

`gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID3`:     0:17.73 (12.80 user,

4.68 sys) 9380k maxres

`gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID3`:     0:40.33 ( 3.70 user,

34.58 sys) 14576k maxres

`g10/gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID3`: 0:16.38 (11.03 user,

4.85 sys) 9280k maxres

`g10/gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID3`: 0:17.83 ( 2.16 user,

13.79 sys) 14612k maxres

• $keyID4: 1439 sigs (643 unique signers), 1168 from an unknown signer (555

unique unknown signers)

`gpg --homedir /tmp/gnupg1 --list-sigs $keyID4`:      0:08.94 (6.64 user,

2.22 sys) 7952k maxres

`gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID4`:     0:09.61 (7.26 user,

1.93 sys) 8444k maxres

`gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID4`:     1:34.20 (5.98 user,

85.03 sys) 11724k maxres

`g10/gpg2 --homedir /tmp/gnupg1 --list-sigs $keyID4`: 0:08.50 (6.37 user,

1.94 sys) 8092k maxres

`g10/gpg2 --homedir /tmp/gnupg2 --list-sigs $keyID4`: 0:50.71 (3.20 user,

44.63 sys) 11568k maxres

Furthermore, the patch brings down --list-sigs on the entire keybox to 17min
from 2h25 (vs 2min on a pubring.gpg).

So all in a all, while this patch significantly improves the slowness of
--list-sigs with a keybox, it doesn't generally (except on $keyID3) make it as
fast as when used with the legacy pubring.gpg format.

Also, I'd like to point out that since the nested lookups in get_user_id are
only meant to retrieve the signer's primary UID, and as it seems to be part of
the keybox's metadata (if I got T1939 right, that's why kbxutil is lightning
fast), it should be possible avoid the parsing (which I guess is causing most of
the overhead) to get the UID right? Of course it can't hurt to have a caching
mechanism on top of that, though ;-)

Done in c238340:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=c2383407bba5eefea486464a31e02846124c9da5

This was fixed by:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=6619ead2cfd2abcb95b66dc70622fdeef624fb8a

using the test described in T1921 (aheinecke on Mar 10 2015, 06:13 PM / Roundup) there are no longer duplicated certificates
shown after the verify command.

Thanks!

I can't reproduce this problem neither in our company setup nor on a vanilla debian.

I've placed the .der files in the correct directories
/var/lib/dirmngr/extra-certs and /etc/dirmngr/trusted-certs

gpgsm --import aheinecke.der

Dirmngr output shows that the LOOKUP Issuer and Intermediate -Cert are not not
found in the dirmngr cache and they are not imported into the keyring.

This is probably another bug that hid this issue in the past.

Right, this is simply done by multiplying by 365. I would consider thuis a
minor bug. Should we add a priority "minor bug" to this tracker?

Thank you for further information.
Now, I understand your situation of mixture of architectures.
I think that your source code was once configured by 32-bit environment (which
created links to 32-bit), and then you tried to configure and to compile by
64-bit environment which caused errors.

I think that "make distclean; configure; make" would success even on the 32-bit
environment with different host OS.

libgcrypt compiles fine on regular FreeBSD 10.1 with the proper
architecture. The problem with the mismatched architecture was because
I was on a virtual machine and I had to run a 32 bit OS in the machine
because my 64-bit hardware didn't have hardware virtualization and
would only run a 32-bit OS. I guess that situation isn't too common,
but I'll leave it to you to classify the original report as a bug or
converted feature request/idea.

Thanks for the information.
I think the complete IDNA and co are an big mining field until to day.

OpenPGP requires the use of UTF-8. OpenPGP does not even know about mail or IDNA.
The translation from UTF-8 needs to be done in your MUA.

By keeping track of not-found user ids I was able to to reduce the time for
--list-sigs on my test setting from more than 5 minutes to 50 seconds. To
properly implement this I need to write a more generic caching mechanism so that
it is only used during key listing but not using import. Using a hash table
instead of simple list will also reduce the lookup time a little bit.

It all depends on the order the keys are stored in the pubring.*, how many keys
are missing and on whether the cache must be flushed.

You might want to apply the attached patch and in another test configure gnupg with
--enable-key-cache=100000
or so and test whether this speeds it up for you.

D294: 597_0001-gpg-TEST-with-no-uid-cache.patch

kbxutil merely dumps the meta data stored in the file and does no parsing at
all. It will never be possible to achieve the same speed: The meta data is only
used to quickly search for certain attributes but not to list them in an
appropriate way. I will thus close this issue.

Just to be clear (it isn't clear to me that these issues are the same),
issue1938 is related to --list-sigs being extremely slow and spending all its
time in kernel mode, while in T1939 I merely wish that doing a keyring dump
with --list-keys was as fast as using kbxutils.

Degrade to feature: Better diagnostic.

It is some kind of regression - I'll look into it soon.

That is the same as T1938.

Duplicate of T1938

I know. It is a regression. I will look into it soon.

It is fixed by the commit: f82c4a6d0d76e716b6a7b22ca964fa2da1f962a0
This is not a perfect solution (it updates key storage by "learn --force" command
of gpg-agent), but it works fine usually.

It seems for me that your build environment is not clean and has some links for
i386, while your arch is x86_64. It is i386's mpih-add1 which has ALIGN(3) at
line number 44.

Please do 'make distclean' and configure, then make.

I understand your case.