D351: 776_0001-gpg-Make-ASCII-armor-decoding-more-robust-to-encodin.patch

gpg doesn't normally directly ask for a password. Instead, operations that
require a password are typically handled by gpg-agent, which is a small server
that is started on demand. (Normally, there is only a single gpg-agent per
user.) When gpg-agent needs a password, it invokes a pinentry program. The
default pinentry can be determined using `gpgconf --list-config'. This can be
overridden using the pinentry-program configuration option in gpg-agent.conf.
(If you change that file, you'll need to restart gpg-agent using something like
`gpgconf --reload gpg-agent'.)

There are several different pinentry programs: pinentry-gtk-2, pinentry-qt,
pinentry-curses and pinentry-tty. (pinentry is typically an alias that is
configured by the system's package manager.) Even if you use pinentry-gtk-2, it
will normally fall back the curses backend if there is no X display.

The issue you might be having is that pinentry might be showing up on a
different display / console.

So, I think this might just be a configuration problem. Nevertheless, I
encourage you to investigate some more and try to figure out what is going on
and report back here. Thanks!

This should be fixed in acac103. (I was able to exactly reproduce your problem
and the patch fixed it for me.) If you are able to test and it works for you,
please report back here.

Thanks!

Note, tests work together with patch from T2253

Attached patch for gpg-preset-passphrase. eval_redirect code copied from
libassuna and modifies.
Second patch creates socked redirection for tests if neccesary

D352: 769_gnupg2-2.1.11_redirect_socket_for_tests.patch

D353: 768_gnupg2-2.1.11_gpg_preset_passphrase_socket_redirect.patch

From looking at the strace in T2229 (t8m on Feb 02 2016, 10:32 AM / Roundup) it seems that the client died after
having seen the first PROGRESS line. The client in this case is
gpg-preset-passphrase which is an old hack and not a proper Assuan
client. It received the 'S PROGRESS' line but does not expect it and
returns an error.

We should re-implement that tool in terms of gpg-connect-agent and for
the test scripts use gpg-connect-agent directly. Justus is currently
reworking the test suite anyway.

It seems like detecting and correcting this simple manging would be
straightforward to do and relatively self contained.

Patch looks good, thanks!

My mistake. I was talking about gpg-preset-passphrase.
Redirect in gpg-agent works as expected.

Sorry, here it is.

D347: 767_0001-g10-Make-sure-to-have-the-directory-for-trustdb-v3.patch

I think you attached the original patch once again by mistake...

D348: 766_0001-g10-Make-sure-to-have-the-directory-for-trustdb-v3.patch

This would be final version.
If !create, we can let it return earlier.

You are right to have the check against CREATE.
I'll include that check.

I found the following comment above g10/keyedit.c' fix_key_signature_order
function, which is responsible for printing the "moving a key signature to the
correct place" lines upon --edit-key.

  /*
   * There are some keys out (due to a bug in gnupg), where the sequence
   * of the packets is wrong.  This function fixes that.
   * Returns: true if the keyblock has been fixed.
   *        
   * Note:  This function does not work if there is more than one user ID.
   */

When was the mentioned bug introduced and fixed? git blame tells me the comment
predates Thu Jun 5 07:14:21 2003 +0000, date when cvs2svn created branch
'GNUPG-1-9-BRANCH' (commit 72503314). But Lucas' key was generated much later,
on 2009-07-01.

Of the 100 keys with lowest MSD http://pgp.cs.uu.nl/doc/top_1000.html, 27 have
badly ordered packets; 26 of these have multiple UIDs; and only 20 of the 120
uids with a creation date have been generated before 2009-07-01.

Is there a technical reason which "This function does not work if there is more
than one user ID"? (I guess gpg could try to verify the sig against each
UID/UAT to find out which one it binds to.) Or is Luca's key (among many
others) broken from a WoT perspective due to the multiple UIDs?

D349: 765_0001-g10-Make-sure-to-have-the-directory-for-trustdb-v2.patch

Thanks for the patch, but it still needs a small change. You don't want to
create the directory or the lock file if the user specified --no-auto-check-trustdb.

How about this patch?

Please try attached patch.
While I understand it's a regression (and it's urgent for you), I downgrade the
priority.
It will be soon in the repo.

D350: 764_0001-g10-Make-sure-to-have-the-directory-for-trustdb.patch

Thank you for the report. Confirmed. It was my mistake, I didn't test the code
path with no homedir.

I'm also interested in this, since i want to make it possible to easily build a
win32 version of gpgv.exe on debian systems. This is possible without iconv at
all in 1.4.x, but i would rather we ship a gpgv from 2.1.x in the future.

Why is this a reasonable assumption? This proposal changes the way that GnuPG
has been working for years and will inevitably break someone's setup. It would
be much better for the receiver to use a non-critical notation to indicate the
desired behavior.

D345: 761_0001-gpg-Fix-calc_header_length-when-LEN-is-0-and-improve.patch

Since it was so trivial to create, I've attach an alternative patch with my
proposed change. Please let me know which one to apply.

D346: 760_0001-gpg-Fix-documentation-for-calc_header_length.patch

Patch attached. Is this okay to apply?

Needs to be documented. I see no reason to change this because because it has
no effect.

Also the pinentry.sh script does not look like being called during the opengpg
tests at all because I've added 'exit 1' directly to the beginning of it and
nothing changed even with the gnupg-2.1.10 make check which passed.

This is what I see in strace log from the gpg-agent during the test - so it is
related to addition of the progress messages.

26074 read(4, "PRESET_PASSPHRASE 50B2D4FA4122C2"..., 1002) = 69
26074 getrusage(RUSAGE_SELF, {ru_utime={0, 0}, ru_stime={0, 2622}, ...}) = 0
26074 clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {0, 2652041}) = 0
26074 write(4, "S PROGRESS open_dev_random X 1 0", 32) = 32
26074 write(4, "\n", 1) = 1
26074 open("/dev/urandom", O_RDONLY) = 5
26074 fcntl(5, F_GETFD) = 0
26074 fcntl(5, F_SETFD, FD_CLOEXEC) = 0
26074 write(4, "S PROGRESS need_entropy X 60 120", 32) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 poll([{fd=5, events=POLLIN}], 1, 0) = 1 ([{fd=5, revents=POLLIN}])
26074 read(5,
"\224l\240\r\205PGH:;\227\370pv\355\202df\24\201\250\272p\257\334\2\304Z\177W\244Q"...,

60. = 60

26074 write(4, "S PROGRESS need_entropy X 120 12"..., 33) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 write(4, "S PROGRESS need_entropy X 60 120", 32) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 poll([{fd=5, events=POLLIN}], 1, 0) = 1 ([{fd=5, revents=POLLIN}])
26074 read(5,
"\222\251\303;\247\377\302Z\t[\10\354\217\236\357?\323\246\210]+\330\341\335*7\315\17\230\3141\211"...,

60. = 60

26074 write(4, "S PROGRESS need_entropy X 120 12"..., 33) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 write(4, "S PROGRESS need_entropy X 60 120", 32) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 poll([{fd=5, events=POLLIN}], 1, 0) = 1 ([{fd=5, revents=POLLIN}])
26074 read(5,
"}\37\0267k\343DGi\372\r&\3El\305\223\312|\307\200U6\24RI\6\214\4H\273\377"...,

60. = 60

26074 write(4, "S PROGRESS need_entropy X 120 12"..., 33) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 write(4, "S PROGRESS need_entropy X 60 120", 32) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 poll([{fd=5, events=POLLIN}], 1, 0) = 1 ([{fd=5, revents=POLLIN}])
26074 read(5,
"\21%\26k\326\1\232\204K\r\33\216\211\1\253;\324\346\362\203?g\22\315\205\203G\344AZ\272\270"...,

60. = 60

26074 write(4, "S PROGRESS need_entropy X 120 12"..., 33) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 getrusage(RUSAGE_SELF, {ru_utime={0, 0}, ru_stime={0, 2971}, ...}) = 0
26074 clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {0, 2978843}) = 0
26074 write(4, "OK", 2) = -1 EPIPE (Broken pipe)
26074 --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26040, si_uid=1000} ---
26074 write(2, "gpg-agent[26040]: Assuan process"..., 55) = 55
26074 write(2, "\n", 1) = 1
26074 close(4) = 0

English original (key info anonymized):

gpg2 --edit-key [keyID]
[version info, copyright/license info]

This key may be revoked by RSA key [keyID] [?]
This key may be revoked by RSA key [keyID] [?]
This key may be revoked by RSA key [keyID] [?]
pub [key size]/[keyID] created: [creation date] expires: [date of expiry]
usage: SC

trust: unknown       validity: unknown

[ trust] (1). [uid]

Misleading German translation:

gpg2 --edit-key [keyID]
[version info, copyright/license info]

Dieser Schlüssel könnte durch RSA mit Schlüssel [keyID] [?] widerrufen worden sein
Dieser Schlüssel könnte durch RSA mit Schlüssel [keyID] [?] widerrufen worden sein
Dieser Schlüssel könnte durch RSA mit Schlüssel [keyID] [?] widerrufen worden sein
pub [key size]/[keyID] erzeugt: [creation date] verfällt: [date of expiry]
Aufruf: SC

Vertrauen: unbekannt     Gültigkeit: unbekannt

[ trust] (1). [uid]

Improved German translation:

Dieser Schlüssel kann von RSA-Schlüssel [keyID] [?] widerrufen werden
...

I'm happy to see GnuPG moving to an all-agent model, where the passphrase and
the asymmetric secret key material aren't available to the gpg process.

That sai, if gpgme is going to remove the passphrase_cb prompt, or to deprecate
it in all cases other than symmetric data encryption/decryption, then should the
API change?

gpgme_set_passphrase_cb is used in about 40 packages in debian:

  https://codesearch.debian.net/results/gpgme_set_passphrase_cb/page_0

this includes bindings for python, ruby, php, and c++ -- and it's possible that
those bindings themselves have some other usage elsewhere.

Do we have guidance for users of this function, whether it's with gpgme
directly, or with any of the bindings?

I have the same problem when building gnupg2 on Fedora 23. Let me know if I can
help with debugging it.

Thanks. This seems to be a gpg 1.4 only bug.

Werner Koch via BTS:

Werner Koch <wk@gnupg.org> added the comment:

Sorry, the logs do not help very much. There is a problem with the pjnentry
which for the tests is a simple script and not the configured one. We need to
replicate the failure to debug it.

I have tested this bug in Debian and Windows.

When running "gpg --gen-key --expert" GPG displays:

     DSA keys may be between 512 and 3072 bits long.

and

     ELG-E keys may be between 512 and 4096 bits long.

however entering 512 will result in

     gpg: keysize invalid; using 2048 bits

     gpg --gen-key --expert
     gpg (GnuPG) 1.4.20; Copyright (C) 2015 Free Software Foundation, Inc.
     This is free software: you are free to change and redistribute it.
     There is NO WARRANTY, to the extent permitted by law.

     Please select what kind of key you want:
        (1) RSA and RSA (default)
        (2) DSA and Elgamal
        (3) DSA (sign only)
        (4) RSA (sign only)
        (7) DSA (set your own capabilities)
        (8) RSA (set your own capabilities)
     Your selection? 2

--> DSA keys may be between 512 and 3072 bits long.

What keysize do you want? (2048) 512
Requested keysize is 512 bits

--> ELG-E keys may be between 512 and 4096 bits long.

     What keysize do you want for the subkey? (2048) 512
     Requested keysize is 512 bits
     Please specify how long the key should be valid.
              0 = key does not expire
           <n>  = key expires in n days
           <n>w = key expires in n weeks
           <n>m = key expires in n months
           <n>y = key expires in n years
     Key is valid for? (0) 0
     Key does not expire at all
     Is this correct? (y/N) y

     You need a user ID to identify your key; the software constructs the user ID
     from the Real Name, Comment and Email Address in this form:
         "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

     Real name: user_id
     Email address:
     Comment:
     You selected this USER-ID:
         "user_id"

     Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
     You need a Passphrase to protect your secret key.

     You don't want a passphrase - this is probably a *bad* idea!
     I will do it anyway.  You can change your passphrase at any time,
     using this program with the option "--edit-key".

     We need to generate a lot of random bytes. It is a good idea to perform
     some other action (type on the keyboard, move the mouse, utilize the
     disks) during the prime generation; this gives the random number
     generator a better chance to gain enough entropy.

--> gpg: keysize invalid; using 2048 bits

gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest

size

...[truncated]...
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

--> gpg: keysize invalid; using 2048 bits

     ...[truncated]...

     gpg: key F0E7A41B marked as ultimately trusted
     public and secret key created and signed.

     gpg: checking the trustdb
     gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
     gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
     pub   2048D/F0E7A41B 2016-02-01
           Key fingerprint = C789 E572 4A8B BC1B 3108  F34E 36F4 D0CC F0E7 A41B
     uid                  user_id
     sub   2048g/977768CF 2016-02-01

Please try 2.1.11 - we fixed something in regard to this.

Sorry, the logs do not help very much. There is a problem with the pjnentry
which for the tests is a simple script and not the configured one. We need to
replicate the failure to debug it.

Jens,
thanks for the report. Now I can classify this as GnuPG "modern" issue. :)
Bernhard

Please explain what you are eactly doing: Give the command typed and all output.
What OS are you using.

In fact Luca' key can currently be found on the keyserver pool with badly
ordered packets (I can provide a copy if need be):

~$ gpg2 --homedir="$gnupghome" --keyserver-options import-minimal --keyserver

hkp://pool.sks-keyservers.net --recv-key "$key"

~$ gpg2 --homedir="$gnupghome" --with-colons --list-sigs "$key" | grep -E

'^(pub|sub|uid|sig:([^:]*:){3}(06EAA066E397832F|39278DA8109E6244)):'

pub:-:4096:1:06EAA066E397832F:1246459499:::-:::scESCA:::::::
uid:-::::1286747091::B41FA634ADD68A6717D380A790190CB3BC80005B::Luca Capello

<luca@pca.it>:::::::::

sig:::1:06EAA066E397832F:1286747091::::Luca Capello <luca@pca.it>:13x:::::8:
uid:-::::1286747538::3590ECEB44695F2B0D4E5B2E85EDBBF99C3A90C6::Luca Capello

<gismo@debian.org>:::::::::

sig:::1:06EAA066E397832F:1286747538::::Luca Capello <luca@pca.it>:13x:::::8:
uid:-::::1453646682::8523545E8C0C86F63F6FC3387DE2D188A55481AF::Luca Capello

<luca.capello@infomaniak.ch>:::::::::

sig:::1:06EAA066E397832F:1453646682::::Luca Capello <luca@pca.it>:13x:::::10:
uid:-::::1454107799::45C4E00E6D5D53EDE22B1CC8D2B44DCE3E3E93B5::Luca Capello

<luca.capello@infomaniak.com>:::::::::

  sig:::1:06EAA066E397832F:1454107799::::Luca Capello <luca@pca.it>:13x:::::10:
  sub:-:4096:1:90C02DEC2BB95F4B:1246460155::::::e::::::
  sig:::1:06EAA066E397832F:1246460155::::Luca Capello <luca@pca.it>:18x:::::8:
  sub:-:4096:1:D91D57A03BE9F36D:1246460943::::::esa::::::
  sig:::1:39278DA8109E6244:1360031056::::[User ID not found]:10x:::::10:
  sig:::1:06EAA066E397832F:1246459499::::Luca Capello <luca@pca.it>:13x:::::8:
  sig:::1:06EAA066E397832F:1246460297::::Luca Capello <luca@pca.it>:13x:::::8:
  sig:::1:06EAA066E397832F:1286747091::::Luca Capello <luca@pca.it>:13x:::::8:
  sig:::1:06EAA066E397832F:1246460943::::Luca Capello <luca@pca.it>:18x:::::8:

Is there any reason why --import/--recv-key didn't move the packets to their
proper place? After all the keyring is then open in write mode.

Moreover while --edit attempts to reorder the packets, it places the signature
packets under the wrong UID:

~$ gpg2 --homedir="$gnupghome" --edit "$key" save
[…]
gpg: moving a key signature to the correct place
~$ gpg2 --homedir="$gnupghome" --with-colons --list-sigs "$key" | grep -E

'^(pub|sub|uid|sig:([^:]*:){3}(06EAA066E397832F|39278DA8109E6244)):'

pub:-:4096:1:06EAA066E397832F:1246459499:::-:::scESCA:::::::
uid:-::::1286747091::B41FA634ADD68A6717D380A790190CB3BC80005B::Luca Capello

<luca@pca.it>:::::::::

sig:::1:06EAA066E397832F:1286747091::::Luca Capello <luca@pca.it>:13x:::::8:
uid:-::::1286747538::3590ECEB44695F2B0D4E5B2E85EDBBF99C3A90C6::Luca Capello

<gismo@debian.org>:::::::::

sig:::1:06EAA066E397832F:1286747538::::Luca Capello <luca@pca.it>:13x:::::8:
uid:-::::1453646682::8523545E8C0C86F63F6FC3387DE2D188A55481AF::Luca Capello

<luca.capello@infomaniak.ch>:::::::::

sig:::1:06EAA066E397832F:1453646682::::Luca Capello <luca@pca.it>:13x:::::10:
uid:-::::1454107799::45C4E00E6D5D53EDE22B1CC8D2B44DCE3E3E93B5::Luca Capello

<luca.capello@infomaniak.com>:::::::::

  sig:::1:06EAA066E397832F:1454107799::::Luca Capello <luca@pca.it>:13x:::::10:
  sig:::1:06EAA066E397832F:1286747091::::Luca Capello <luca@pca.it>:13x:::::8:
  sig:::1:06EAA066E397832F:1246460297::::Luca Capello <luca@pca.it>:13x:::::8:
  sig:::1:06EAA066E397832F:1246459499::::Luca Capello <luca@pca.it>:13x:::::8:
  sig:::1:39278DA8109E6244:1360031056::::[User ID not found]:10x:::::10:
  sub:-:4096:1:90C02DEC2BB95F4B:1246460155::::::e::::::
  sig:::1:06EAA066E397832F:1246460155::::Luca Capello <luca@pca.it>:18x:::::8:
  sub:-:4096:1:D91D57A03BE9F36D:1246460943::::::esa::::::
  sig:::1:06EAA066E397832F:1246460943::::Luca Capello <luca@pca.it>:18x:::::8:

I (0x39278DA8109E6244) did *not* sign Luca's 4th UID. I'm unsure (based on
--list-packets' output) which of the 2 first UIDs my signature applies to, but
certainly not to the last two, which were created 3 years after my sig was
issued.

In fact this is reproducible with Luca's key (but strangely not with mine):

~$ gpg2 --version
gpg (GnuPG) 2.1.11
~$ gnupghome=$(mktemp -d)
~$ key=0x06EAA066E397832F
~$ gpg2 --homedir="$gnupghome" --keyserver hkp://pool.sks-keyservers.net

--recv-key "$key"

~$ gpg2 --homedir="$gnupghome" --edit-key "$key" minimize 4 deluid save
~$ gpg2 --homedir="$gnupghome" --keyserver hkp://pool.sks-keyservers.net

--recv-key "$key"

  ~$ gpg2 --homedir="$gnupghome" --with-colons --list-sigs "$key"

The last command shows a lot of signatures under the last subkey. This not only
messes up the parsing, but also confuses GnuPG: for instance it refuses to let
me sign the 4th UID because it thinks I already did.

Hi Bernhard,

all concatenated tests 2.1.10 by contrast
hope it helps

all concatenated tests

This is likey due to the card already decoding the pkcs#1 - we need to look
closer at this use case.

For reference, I have a OpenPGP v2.0 card from "ZeitControl".

I think the card will always remove the encoding internally and only return the
plaintext, as far as I can tell from
http://g10code.com/docs/openpgp-card-2.0.pdf, Section 7.2.9

check out tests/openpgp/version.test.log or the oter *test.log files.

Look here:

gpgsm: DBG: pkcs1 encoded session key: 11 E8 C4 40 93 A8 24 35 16 57 93 8D 03 00
63 5F
gpgsm: decrypting session key failed: Invalid session key

This is clearly not a PKCS#1 encoded session key but a plain session key. This
is likey due to the card already decoding the pkcs#1 - we need to look closer at
this use case.

Hi Jens,
which version of gpg2 on which platform did you try this?

Which version of gnupg2 do you refer to? (On which platform?)
AFAIK 2.0.29 gpg2 does not have a --faked-system-time option.