Thanks a lot. I will test as soon as you release the test build.

Hi,

Thanks for testing gpg4win This issue was already reported in T2335 and has
been resolved (but not yet released).

I'll upload a new beta next week.

Regards,
Andre

Still true for sending but for sending we don't have a choice. But decryption is
now done in a different thread.

I've tried this again with the current development version after a very large
refactoring how we handle mails. The bug appears to be gone. I've tested 10
times to send a file with closed / open outlook and with and without encryption
active.

If I install gpg4win-2.3.3 on the same system / setup the crash is reliably
reproducible.

It's still likely that we made a reference counting error internally in code
that was changed / fixed now. And Outlook released the Mail object too early and
crashed.

Kaspersky probably had some similar error in their code.

I'll upload a new Gpg4win beta with the new gpgol next. I'll ping in this issue
once thats done so you could ideally confirm that its fixed now.

That is a bit complicated and would require new strings. I do not think that is
justified.

I don't think there is an issue anymore.

Still in issue after we figured out that select could have been the reason for
this? The explict check for a valid FD number was added in June so before that
a segv could have been the outcome.

With the soon to be released gnupg 2.1.16 and gpgme 1.8 this will work.

If a keyring is non-readable (e.g. chmod 000 pubring.bkx), gpgme_op_keylist_next
will return GPG_ERR_OPENKEYRING instead of GPG_ERR_EOF.

The difference (according to the gpg agent log) is that gpg v1 is obviously caching
the decrypted private key used to decrypt the files using the option "-d --
multifile" whereas gpg v2 in my case repeatedly requests the decryption of the
private key for each single file. Any way to change that?

A little bit better, but that would still confuse me, as I did not intentionally
specify an elliptic curve.
What could help here is:

1. talking about algo/algorithm (that is shown in the man page as parameter for

--quick-gen-key)

2. saying which algorithm gpg saw.

If the error message had been "Unkown algo 'user@example.com'" I would
immediately know that I provided an email address where an algorithm was expected.

okay, changed to

Unknown elliptic curve

Andre said, category dirmngr is better

I'm also seeing this behavior when there is something wrong with the reverse DNS
lookups. For example:

Nov 08 10:54:36 alice dirmngr[1714]: handler for fd 5 started
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> # Home: /home/dkg/.gnupg
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> # Config:
/home/dkg/.gnupg/dirmngr.conf
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK Dirmngr 2.1.15 at your
service
Nov 08 10:54:36 alice dirmngr[1714]: connection from process 7623 (1000:1000)
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- GETINFO version
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> D 2.1.15
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- KEYSERVER
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> S KEYSERVER
hkps://hkps.pool.sks-keyservers.net
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- KS_GET --
0x2E8DD26C53F1197DDF403E6118E667F1EB8AF314
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L3: ASSERT:
mpi.c[_gnutls_x509_read_uint]:246
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]:
Allocating epoch #0
Nov 08 10:54:36 alice dirmngr[1714]: can't connect to 'oteiza.siccegge.de':
Invalid argument
Nov 08 10:54:36 alice dirmngr[1714]: error connecting to
'https://oteiza.siccegge.de:443': Invalid argument
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]: Start
of epoch cleanup
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]: End of
epoch cleanup
Nov 08 10:54:36 alice dirmngr[1714]: DBG: gnutls:L5: REC[0x7f7458003000]: Epoch
#0 freed
Nov 08 10:54:36 alice dirmngr[1714]: command 'KS_GET' failed: Invalid argument
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> ERR 167804976 Invalid
argument <Dirmngr>
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 <- BYE
Nov 08 10:54:36 alice dirmngr[1714]: DBG: chan_5 -> OK closing connection
Nov 08 10:54:36 alice dirmngr[1714]: handler for fd 5 terminated

This appears to be because the pool included 92.43.111.21, which has a PTR of
oteiza.siccegge.de, despite the fact that oteiza.siccegge.de has no A record.

There is no reason for dirmngr to be talking to the member of the pool by its
hostname, anyway -- it should make the connection by IP address, with the TLS
SNI set to the pool name.

The TCP specs demand something different and it is not the duty of dirmngr to do
something about it. You have ths behavour with all TCP connections and that is
also what makes TCP a reliable connection.

On Linux if would be possible to reduce the intial SYN retries but that is not
portable.

For --auto-key-retrieve I already implemented a --quick parameter in gpg to
advise dirmngr to give up earlier. The dirmngr side has not been implemented,
though.

There are two related problem, which are only related to the key listing:

We do not indicate in the output whether a user id is valid. Instead we show
the validity info from the trustdb regardless of the time conflict. Ths could
be changed for example to show "[invalid]" instead of "[full]". This covers all
cases which render a user id invalid and not just a time conflict.

Due to the invalid user id the key is also not valid but we do not indicate this
either.

By using --ignore-time-conflict the problem goes away but that is not a
solution. We need to properly indicate when a user id or Key is not valid even
when not doing --check-sigs. One way to do this would be to use the same tags
we use with --checks-sigs with -k for the used self-signatures. That
information is readily available.

Fixed in 5840353d8bbcd9e75374f3bdb2547ffa7bbea897.

Neal, that is exactly what happens, thanks for writing it out.

Werner, yes, it also affects gpg1:

% faketime "2016-07-01" g10/gpg --edit foo
gpg (GnuPG) 1.4.22-beta2; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: key 0707DEE4 was created 29 seconds in the future (time warp or clock problem)

pub 2048R/0707DEE4 created: 2016-06-30 expires: never usage: SCEA

trust: unknown       validity: unknown

[ unknown] (1). foo bar <foo@example.org>
% faketime "2016-07-02" g10/gpg --edit foo
gpg (GnuPG) 1.4.22-beta2; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!

pub 2048R/0707DEE4 created: 2016-06-30 expires: 2016-09-28 usage: C

trust: unknown       validity: unknown

[ unknown] (1). foo bar <foo@example.org>

Because it took me a while to understand what is actually going wrong, a summary
of the problem: if we get an error such as "key 517912BA66E730CA was created 78
seconds in the future", then the key's flags will be wrong (below: SCEA instead
of C) and the expiration date will not be printed.

Interesting stuff. My solution wouyld be to switch to the gtk pinetry, but I'll
take care care of your patch tomorrow.

D386: 903_0001-gnome3-Fall-back-to-curses-if-screensaver-is-locked.patch

Attached is a patch to check for locked screensaver and fall back to curses if
detected.

Perhaps gcr needs to refuse to prompt in the event that the graphical session is
known-idle/locked (in screensaver mode, whatever). Then the pinentry could know
to fall back to the tty because of the locked screen.

I just spent a while trying to research this, and i'm afraid that the code i've
written to detect whether gcr is available does nothing to detect whether the
screen is currently locked.

Furthermore, when "getpin" is called against a dbus session that is locked, it
immediately returns with a "Cancelled" message, in a way that is pretty
difficult to diagnose.

However, it looks like i can query the gnome screensaver via dbus to see whether
the screen is locked. From the command line, that's:

dbus-send --print-reply=literal --session --dest=org.gnome.ScreenSaver

/org/gnome/ScreenSaver org.gnome.ScreenSaver.GetActive

which returns a boolean true or false depending on whether the screen is locked.

We'd just need to translate it into GDBus, i think, perhaps using something
higher-level like g_dbus_connection_call(), or something lower-level, like
g_dbus_connection_send_message_with_reply() (or their synchronous variants):

file:///usr/share/doc/libglib2.0-doc/gio/GDBusConnection.html#g-dbus-connection-call
file:///usr/share/doc/libglib2.0-doc/gio/GDBusConnection.html#g-dbus-connection-send-message-with-reply

In your example, i don't think updatestartuptty is necessary for text-mode
prompting -- the "gpg --decrypt …" process will be able to detect which tty it
is connected to and pass it to the agent.

But the question here has to do with graphical consoles as well, and i don't
think there's a clear answer yet.

There are two X11 graphical sessions in the example:

a) the local machine's graphical console, where the user is currently sitting,
running ssh *to* the remote machine
b) the remote machine's graphical console, where the user is logged in, but idle

There are also three kinds of pinentry user-attention-getting mechanisms:

0) terminal

1. X11
2. d-bus

finally, i'll note that there are (at least) two d-bus user sessions running in
this example: on the remote host and on the local host. I'm assuming in this
example that the user has a single shared d-bus session across all logins on the
computer (this is the dbus-user-session model, which is well-aligned with the
gpg-agent standard-socket model, where there is one running process per user per
machine)

Since "ssh -X remote" forwards the X11 session but not the d-bus session, any
d-bus-based pinentry (like pinentry-gnome3) will connect to the d-bus session on
the remote machine. But the d-bus session on the remote machine is *also*
connected to the remote graphical (X11) console.

pinentry on the remote machine has two choices:

x) talk to the d-bus session it is connected to (which will trigger a prompt on
the remote graphical console, or
y) fall back to curses

If it chooses (x) then the user is unlikely to see the prompt (they're not
sitting in front of that graphical console). But it's not clear how to
distinguish the situation from normal use in order to choose (y).

Perhaps gcr needs to refuse to prompt in the event that the graphical session is
known-idle/locked (in screensaver mode, whatever). Then the pinentry could know
to fall back to the tty because of the locked screen. If it does that, then the
error case (where the graphical prompt is shown on the idle session) is limited
to situations where the user left the remote graphical console unlocked. I
don't know whether we can get gcr to report that successfully or not, though.

They need to run

gpg-connect-agent updatestartuptty /bye

to tell gpg-agent where to open the Pinentry. Depending on how they log in
either a curses or GUI Pinentry will be shown. I.e.

  ssh -X example.org
  gpg-connect-agent updatestartuptty /bye
  gpg --decrypt ....

shows a GUI Pinentry. If -X is not used the curses pinentry comes up.

Not quite true. As soon as a blocking system cal is used another thread is
scheduled. Long running operations like generating a new key may indeed take a
long time and inhibit other threads from running. They run long becuase they
need to collect entropy. Having other threads running at that time would not
really be helpful. Using gpg-agent for more than a decade now, I never made
that experience.

The more likely reason for the problem is that no working pinentry is installed
and the boths threads are waiting for the pinentry (pinentry access is obviously
serialized).

We need a log file from gpg-agent: Out this into gpg-agent.conf

log-file /tmp/foo/agent.log
debug 1024
verbose

and restart the agent.

In gpg-agent, only a single thread of execution runs at a time. So it is
entirely possible that what you are describing happens. For us to debug it, we
need a very concrete example. Please provide us with the command line(s) that
you are using to decrypt the files in parallel. Also, please list the keys. (A
small guess: you are using 16k RSA.)

Can you test this also with 1.4 (iirc, Debian has a tool to fake the sytsem time
for a process)

Fixed with commit df08a0c. Thanks.

I just tried:

$ g10/gpg --encrypt -r samuel </dev/urandom >/dev/null

As expected, the gpg process eats a lot of cpu time, and I can spawn two of them
just fine. This works with both my build as well as gpg from Debian testing.

I once thought about making yatm emit org mode. Wdyt?

I'm closing this bug due to inactivity. Feel free to reopen it with more
information.

Fixed in 60ad1a7f37ffc10e601e69a3e2d2bb14af510257.

Hi Andre,

Thanks for following up. I seem to be able to reproduce the first part of your
issue here and I'm looking in to it.

Thanks,
Neal

Sry I accidentally posted an incomplete message with T2812 (aheinecke on Oct 31 2016, 05:08 PM / Roundup) (I used itsalltext
and postet a wrong version).

I wanted to write:

On the command line it's looking good. The second keylist is also down to 5
seconds on Windows.

But used from gpgme it still takes about a minute. If you add --with-colons and
slow down system calls by using strace you can also see this on GNU/Linux:

~> time strace gpg2 --no-default-keyring \

--keyring /usr/share/keyrings/debian-keyring.gpg \
--no-auto-check-trustdb --trust-model pgp \
--with-colons -k >/dev/null 2>&1

2.26s user 0.40s system 102% cpu 2.601 total
~> time strace gpg2 --no-default-keyring \

--keyring /usr/share/keyrings/debian-keyring.gpg \
--no-auto-check-trustdb --trust-model tofu \
--with-colons -k >/dev/null 2>&1

21.43s user 24.47s system 108% cpu 42.451 total

On Windows it's:

PS C:\Users\aheinecke> Measure-Command -Expression { gpg --no-auto-check-trustdb
--trust-model tofu --list-keys > $null
}
TotalSeconds : 7.0945596

PS C:\Users\aheinecke> Measure-Command -Expression { gpg --no-auto-check-trustdb
--with-colons --trust-model tofu --lis
t-keys > $null }
TotalSeconds : 56.0914993

PS C:\Users\aheinecke> Measure-Command -Expression { gpg --no-auto-check-trustdb
--with-colons --trust-model pgp --list
-keys > $null }
TotalSeconds : 1.4855689

I'm also still seeing decryption blocked on Windows while a keylist
--with-colons runs.

I wonder if we should generally check out performance of reading the keyring on
Windows
mabye we could genrally improve it so that it's better cached by Windows.

No both have unknown trust.

7a634e48b13c5d5d295b8fed9b429e1b2109a333 should fix the contention issue.
Please let me know if you are still having issues.

(see on-list discussion at
https://lists.gnupg.org/pipermail/gnupg-users/2016-October/056978.html)

eec365a & 614ca00 fixed the performance issue for me here.

us@chu:~/neal/work/gpg/test (GnuPGTest)$ rm tofu.db
us@chu:~/neal/work/gpg/test (GnuPGTest)$ time gpg --no-default-keyring --keyring
/usr/share/keyrings/debian-keyring.gpg -k >/dev/null
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: Note: signatures using the MD5 algorithm are rejected

real 0m45.569s
user 0m34.316s
sys 0m10.872s
us@chu:~/neal/work/gpg/test (GnuPGTest)$ time gpg --no-default-keyring --keyring
/usr/share/keyrings/debian-keyring.gpg -k >/dev/null
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: Note: signatures using the MD5 algorithm are rejected

real 0m2.306s
user 0m2.284s
sys 0m0.020s
us@chu:~/neal/work/gpg/test (GnuPGTest)$ time gpg --no-auto-check-trustdb
--trust-model pgp --no-default-keyring --keyring
/usr/share/keyrings/debian-keyring.gpg -k >/dev/null
gpg: NOTE: THIS IS A DEVELOPMENT VERSION!
gpg: It is only intended for test purposes and should NOT be
gpg: used in a production environment or with production keys!
gpg: please do a --check-trustdb
gpg: Note: signatures using the MD5 algorithm are rejected

real 0m2.261s
user 0m2.248s
sys 0m0.012s

The first time a key is encountered, we need to do a number of checks that
require reading its keyblock. These include checking whether the key is signed
by an ultimately trusted key. So, this cost is pretty much unavoidable, but it
should be a one time thing.

That other gpg processes stall is surprising, and I will investigate this. I
went to a fair amount of trouble to make sure that that doesn't happen in practice.

That the cost is higher on subsequent runs is a bit disconcerting. I will also
investigate this.

Are the two keys that you testing ultimately trusted? If so, then their
validity is good independent of their TOFU policy.

It is a bit unfortunate that the TOFU policy doesn't show this. I will try and
fix this, but it is a bit complicated because when a key's ownertrust is changed
(or a signature is added, etc.), the tofu db is not updated.

Duplicate of T2341

Thanks for your report,

This was already fixed in T2341

Which is currently not yet released. I'm marking this issue here as released
with superseder (duplicate) to keep the tracker clean.

Fixed with: 5579c4b4f

The code was overcomplicated as it was based on a bad assumption about Outlook
which I never questioned myself. We now properly encrypt in the send event so no
need for ticklish threads / callbacks.

Well, I can only say right now that since upgrading to Ubuntu 16.10, the gpg
command now is gnupg v2 by default, and my parallel decryption using
multiple gpg processes does not work any more. "Not working" means there is
only one gpg-agent processes using any CPU at all, and it is using only one
CPU core at 100% for a very long time. Nothing else pops up in top regarding
CPU usage. 75% of the CPU cores remain idle. So my guess is that the gpg-
agent does all of the work and therefore prevents multiple parallel
executions. My conclusions seem pretty obvious to me. But maybe it has to do
with stuff done by some downstream debian or Ubuntu packagers?