We see no such failures for our builds (on El Capitan and now on Sierra). We
use these configure flags for all our builds on macOS:

  ./configure --prefix=/Users/jenkins/prefix/native
  --enable-maintainer-mode \
  --enable-wks-tools --enable-g13 --enable-symcryptrun \
  --enable-gpg2-is-gpg --with-libiconv-prefix=/Users/jenkins/pkg
  'CFLAGS= -D_DARWIN_C_SOURCE=900000L -fPIC' \
  'CXXFLAGS= -D_DARWIN_C_SOURCE=900000L  -fPIC -std=c++11'

--enable-maintainer-mode should not be needed, it is only used to
re-create Makefile when they have been added and to enable more
warning flags.

Thanks for the report. Unfortunately we do not have a regression test key for
--export-ssh-key and thus this bug slipped into the release.

Caused by commit d20107f6d.

OK, I don't care enough to warrant more discussion/work on this.
"Unknown elliptic curve" is already better than "Invalid elliptic curve".

I submitted these patches on the gnupg-devel mailing list in November 2016:

https://lists.gnupg.org/pipermail/gnupg-devel/2016-November/032011.html

These are working for me to keep gpg-agent idle on platforms that support
inotify when the user doesn't use scdaemon, and we're now shipping with them
applied in debian unstable.

D276: 907_0140-agent-Avoid-scheduled-checks-on-socket-when-inotify-.patch

D277: 906_0139-agent-Avoid-tight-timer-tick-when-possible.patch

D278: 905_0138-agent-Allow-threads-to-interrupt-main-select-loop-wi.patch

D279: 904_0137-agent-Create-framework-of-scheduled-timers.patch

The algorithm parser works by checking the known "classic" algorithm and then
assumes that anything else is an ellptic curve. You see that all over the place
where you can enter an algorithm name. Thus there is no way to change this.

To be clear: I want the

• less specific and
• already existing

error message "Unknown algorithm" (instead of "Unknown elliptic curve", which is
not correct in too many situations)

Sorry for the delay in getting back to you on this issue. I think you mean they
have undefined trust (that's what I get here). Undefined trust means "not
enough information for calculation" (from trustdb.h).

Can you clarify what you mean by validity conflict?

The --quick-gen-key command with the additional option is for use by scripts and
they should be able to read the manual.

If you look at the code you should see why it is a lot of work for a bit more
specific error message - we already have way to many messages. I could easily
find dozens of other places where we - in theory - could primt more specific
error messages. That would turn into a neverending story.

I fully support dkg on this. If our downstream is complaining that there is a
problem, then we need to take it seriously. I respect Werner's opinion, but
disagree specifically with the idea that this is only a problem for special
users. I think it will happen to many normal users too.

The string "Unknown algorithm" already exists. Because it is less specific, it
does not indicate that there is a problem regarding support for elliptic curves
here.

That is a bit complicated and would require new strings. I do not think that is
justified.

The difference (according to the gpg agent log) is that gpg v1 is obviously caching
the decrypted private key used to decrypt the files using the option "-d --
multifile" whereas gpg v2 in my case repeatedly requests the decryption of the
private key for each single file. Any way to change that?

A little bit better, but that would still confuse me, as I did not intentionally
specify an elliptic curve.
What could help here is:

1. talking about algo/algorithm (that is shown in the man page as parameter for

--quick-gen-key)

2. saying which algorithm gpg saw.

If the error message had been "Unkown algo 'user@example.com'" I would
immediately know that I provided an email address where an algorithm was expected.

For backward compatibily reasons.

okay, changed to

Unknown elliptic curve

Andre said, category dirmngr is better

There are two related problem, which are only related to the key listing:

We do not indicate in the output whether a user id is valid. Instead we show
the validity info from the trustdb regardless of the time conflict. Ths could
be changed for example to show "[invalid]" instead of "[full]". This covers all
cases which render a user id invalid and not just a time conflict.

Due to the invalid user id the key is also not valid but we do not indicate this
either.

By using --ignore-time-conflict the problem goes away but that is not a
solution. We need to properly indicate when a user id or Key is not valid even
when not doing --check-sigs. One way to do this would be to use the same tags
we use with --checks-sigs with -k for the used self-signatures. That
information is readily available.

Fixed in 5840353d8bbcd9e75374f3bdb2547ffa7bbea897.

Neal, that is exactly what happens, thanks for writing it out.

Werner, yes, it also affects gpg1:

% faketime "2016-07-01" g10/gpg --edit foo
gpg (GnuPG) 1.4.22-beta2; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: key 0707DEE4 was created 29 seconds in the future (time warp or clock problem)

pub 2048R/0707DEE4 created: 2016-06-30 expires: never usage: SCEA

trust: unknown       validity: unknown

[ unknown] (1). foo bar <foo@example.org>
% faketime "2016-07-02" g10/gpg --edit foo
gpg (GnuPG) 1.4.22-beta2; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!

pub 2048R/0707DEE4 created: 2016-06-30 expires: 2016-09-28 usage: C

trust: unknown       validity: unknown

[ unknown] (1). foo bar <foo@example.org>

Because it took me a while to understand what is actually going wrong, a summary
of the problem: if we get an error such as "key 517912BA66E730CA was created 78
seconds in the future", then the key's flags will be wrong (below: SCEA instead
of C) and the expiration date will not be printed.

Interesting stuff. My solution wouyld be to switch to the gtk pinetry, but I'll
take care care of your patch tomorrow.

D386: 903_0001-gnome3-Fall-back-to-curses-if-screensaver-is-locked.patch

Attached is a patch to check for locked screensaver and fall back to curses if
detected.

Perhaps gcr needs to refuse to prompt in the event that the graphical session is
known-idle/locked (in screensaver mode, whatever). Then the pinentry could know
to fall back to the tty because of the locked screen.

I just spent a while trying to research this, and i'm afraid that the code i've
written to detect whether gcr is available does nothing to detect whether the
screen is currently locked.

Furthermore, when "getpin" is called against a dbus session that is locked, it
immediately returns with a "Cancelled" message, in a way that is pretty
difficult to diagnose.

However, it looks like i can query the gnome screensaver via dbus to see whether
the screen is locked. From the command line, that's:

dbus-send --print-reply=literal --session --dest=org.gnome.ScreenSaver

/org/gnome/ScreenSaver org.gnome.ScreenSaver.GetActive

which returns a boolean true or false depending on whether the screen is locked.

We'd just need to translate it into GDBus, i think, perhaps using something
higher-level like g_dbus_connection_call(), or something lower-level, like
g_dbus_connection_send_message_with_reply() (or their synchronous variants):

file:///usr/share/doc/libglib2.0-doc/gio/GDBusConnection.html#g-dbus-connection-call
file:///usr/share/doc/libglib2.0-doc/gio/GDBusConnection.html#g-dbus-connection-send-message-with-reply

In your example, i don't think updatestartuptty is necessary for text-mode
prompting -- the "gpg --decrypt …" process will be able to detect which tty it
is connected to and pass it to the agent.

But the question here has to do with graphical consoles as well, and i don't
think there's a clear answer yet.

There are two X11 graphical sessions in the example:

a) the local machine's graphical console, where the user is currently sitting,
running ssh *to* the remote machine
b) the remote machine's graphical console, where the user is logged in, but idle

There are also three kinds of pinentry user-attention-getting mechanisms:

0) terminal

1. X11
2. d-bus

finally, i'll note that there are (at least) two d-bus user sessions running in
this example: on the remote host and on the local host. I'm assuming in this
example that the user has a single shared d-bus session across all logins on the
computer (this is the dbus-user-session model, which is well-aligned with the
gpg-agent standard-socket model, where there is one running process per user per
machine)

Since "ssh -X remote" forwards the X11 session but not the d-bus session, any
d-bus-based pinentry (like pinentry-gnome3) will connect to the d-bus session on
the remote machine. But the d-bus session on the remote machine is *also*
connected to the remote graphical (X11) console.

pinentry on the remote machine has two choices:

x) talk to the d-bus session it is connected to (which will trigger a prompt on
the remote graphical console, or
y) fall back to curses

If it chooses (x) then the user is unlikely to see the prompt (they're not
sitting in front of that graphical console). But it's not clear how to
distinguish the situation from normal use in order to choose (y).

Perhaps gcr needs to refuse to prompt in the event that the graphical session is
known-idle/locked (in screensaver mode, whatever). Then the pinentry could know
to fall back to the tty because of the locked screen. If it does that, then the
error case (where the graphical prompt is shown on the idle session) is limited
to situations where the user left the remote graphical console unlocked. I
don't know whether we can get gcr to report that successfully or not, though.

They need to run

gpg-connect-agent updatestartuptty /bye

to tell gpg-agent where to open the Pinentry. Depending on how they log in
either a curses or GUI Pinentry will be shown. I.e.

  ssh -X example.org
  gpg-connect-agent updatestartuptty /bye
  gpg --decrypt ....

shows a GUI Pinentry. If -X is not used the curses pinentry comes up.

Not quite true. As soon as a blocking system cal is used another thread is
scheduled. Long running operations like generating a new key may indeed take a
long time and inhibit other threads from running. They run long becuase they
need to collect entropy. Having other threads running at that time would not
really be helpful. Using gpg-agent for more than a decade now, I never made
that experience.

The more likely reason for the problem is that no working pinentry is installed
and the boths threads are waiting for the pinentry (pinentry access is obviously
serialized).

We need a log file from gpg-agent: Out this into gpg-agent.conf

log-file /tmp/foo/agent.log
debug 1024
verbose

and restart the agent.

How many people has this happened to? how many people haven't known to find you
on freenode and ask about it? how many people have just given up on gpg
instead, or just decided "2.1 is broken"?

Shouldn't we fix this for them?

In gpg-agent, only a single thread of execution runs at a time. So it is
entirely possible that what you are describing happens. For us to debug it, we
need a very concrete example. Please provide us with the command line(s) that
you are using to decrypt the files in parallel. Also, please list the keys. (A
small guess: you are using 16k RSA.)

Can you test this also with 1.4 (iirc, Debian has a tool to fake the sytsem time
for a process)

FWIW, I idle on gnupg on freenode and I've helped a bunch of people over the
past two years with exactly this problem. It is not that they want to use gpg
and gpg2, but that at some point they (or some tool) ran gpg2 while they
continued to use gpg1. They then became very surprised when they used gpg2 and
it only had a subset of their keys. My advice for these users is always the
same: remove the migration file and just rerun gpg2. As far as I can tell, this
has fixed the problem in all cases.

I am working on that...

I just tried:

$ g10/gpg --encrypt -r samuel </dev/urandom >/dev/null

As expected, the gpg process eats a lot of cpu time, and I can spawn two of them
just fine. This works with both my build as well as gpg from Debian testing.

I once thought about making yatm emit org mode. Wdyt?

Fixed in ab89164be02012f1bf159c971853b8610e966301.

I also don't quite understand why we restrict this to user ids resembling mail
addresses, so I'll keep issue this open for discussion.

Fixed in 60ad1a7f37ffc10e601e69a3e2d2bb14af510257.

Hi Andre,

Thanks for following up. I seem to be able to reproduce the first part of your
issue here and I'm looking in to it.

Thanks,
Neal

Sry I accidentally posted an incomplete message with T2812 (aheinecke on Oct 31 2016, 05:08 PM / Roundup) (I used itsalltext
and postet a wrong version).

I wanted to write:

On the command line it's looking good. The second keylist is also down to 5
seconds on Windows.

But used from gpgme it still takes about a minute. If you add --with-colons and
slow down system calls by using strace you can also see this on GNU/Linux:

~> time strace gpg2 --no-default-keyring \

--keyring /usr/share/keyrings/debian-keyring.gpg \
--no-auto-check-trustdb --trust-model pgp \
--with-colons -k >/dev/null 2>&1

2.26s user 0.40s system 102% cpu 2.601 total
~> time strace gpg2 --no-default-keyring \

--keyring /usr/share/keyrings/debian-keyring.gpg \
--no-auto-check-trustdb --trust-model tofu \
--with-colons -k >/dev/null 2>&1

21.43s user 24.47s system 108% cpu 42.451 total

On Windows it's:

PS C:\Users\aheinecke> Measure-Command -Expression { gpg --no-auto-check-trustdb
--trust-model tofu --list-keys > $null
}
TotalSeconds : 7.0945596

PS C:\Users\aheinecke> Measure-Command -Expression { gpg --no-auto-check-trustdb
--with-colons --trust-model tofu --lis
t-keys > $null }
TotalSeconds : 56.0914993

PS C:\Users\aheinecke> Measure-Command -Expression { gpg --no-auto-check-trustdb
--with-colons --trust-model pgp --list
-keys > $null }
TotalSeconds : 1.4855689

I'm also still seeing decryption blocked on Windows while a keylist
--with-colons runs.

I wonder if we should generally check out performance of reading the keyring on
Windows
mabye we could genrally improve it so that it's better cached by Windows.

I like this work, thanks for it! I wonder whether it would also be useful for
full-match userID, not only for a raw e-mail address?

For example, if i query for '=Peter Palfrader' or '=ssh://host.example', it
ought to give me the key with the highest-validity binding for the requested
user ID.

No both have unknown trust.

7a634e48b13c5d5d295b8fed9b429e1b2109a333 should fix the contention issue.
Please let me know if you are still having issues.

(see on-list discussion at
https://lists.gnupg.org/pipermail/gnupg-users/2016-October/056978.html)

eec365a & 614ca00 fixed the performance issue for me here.

us@chu:~/neal/work/gpg/test (GnuPGTest)$ rm tofu.db
us@chu:~/neal/work/gpg/test (GnuPGTest)$ time gpg --no-default-keyring --keyring
/usr/share/keyrings/debian-keyring.gpg -k >/dev/null
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: Note: signatures using the MD5 algorithm are rejected

real 0m45.569s
user 0m34.316s
sys 0m10.872s
us@chu:~/neal/work/gpg/test (GnuPGTest)$ time gpg --no-default-keyring --keyring
/usr/share/keyrings/debian-keyring.gpg -k >/dev/null
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: Note: signatures using the MD5 algorithm are rejected

real 0m2.306s
user 0m2.284s
sys 0m0.020s
us@chu:~/neal/work/gpg/test (GnuPGTest)$ time gpg --no-auto-check-trustdb
--trust-model pgp --no-default-keyring --keyring
/usr/share/keyrings/debian-keyring.gpg -k >/dev/null
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: please do a --check-trustdb
gpg: Note: signatures using the MD5 algorithm are rejected

real 0m2.261s
user 0m2.248s
sys 0m0.012s

The first time a key is encountered, we need to do a number of checks that
require reading its keyblock. These include checking whether the key is signed
by an ultimately trusted key. So, this cost is pretty much unavoidable, but it
should be a one time thing.

That other gpg processes stall is surprising, and I will investigate this. I
went to a fair amount of trouble to make sure that that doesn't happen in practice.

That the cost is higher on subsequent runs is a bit disconcerting. I will also
investigate this.

Are the two keys that you testing ultimately trusted? If so, then their
validity is good independent of their TOFU policy.

It is a bit unfortunate that the TOFU policy doesn't show this. I will try and
fix this, but it is a bit complicated because when a key's ownertrust is changed
(or a signature is added, etc.), the tofu db is not updated.

Well, I can only say right now that since upgrading to Ubuntu 16.10, the gpg
command now is gnupg v2 by default, and my parallel decryption using
multiple gpg processes does not work any more. "Not working" means there is
only one gpg-agent processes using any CPU at all, and it is using only one
CPU core at 100% for a very long time. Nothing else pops up in top regarding
CPU usage. 75% of the CPU cores remain idle. So my guess is that the gpg-
agent does all of the work and therefore prevents multiple parallel
executions. My conclusions seem pretty obvious to me. But maybe it has to do
with stuff done by some downstream debian or Ubuntu packagers?