Apparent regressions between 2.2.32 and 2.2.33 of GnuPG
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	vsajip
	Dec 19 2021, 2:33 PM

Description

The unit tests in my project, https://github.com/vsajip/python-gnupg, have started failing following an upgrade to GnuPG 2.2.33. Previously with 2.2.32, there were no failures. With 2.3.3, there are no failures either, so it does seem like a regression. Tests were run on Linux Mint 19.1 (based on Ubuntu bionic) 64-bit.

Below are sections in my test log showing the failures:

Failure 1:

DEBUG gnupg      MainThread  967 27322: gpg2 --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir /tmp/keys-c_alhlfn --debug-quick-random --encrypt --recipient 99BFC3DB10FE4BE821EA1228EF574800D803274C --recipient 9E26CD705E8C31F39A0E7E1C9BFFF293021B7D5E --armor
DEBUG gnupg      MainThread  173 data copier: <Thread(Thread-45 (_copy_data), initial daemon)>, <_io.BytesIO object at 0x7f9d27c5ba10>, <_io.BufferedWriter name=5>
DEBUG gnupg      MainThread 1029 stderr reader: <Thread(Thread-46 (_read_response), initial daemon)>
DEBUG gnupg      Thread-45 (_copy_data)  168 closed output, 13 bytes sent
DEBUG gnupg      MainThread 1036 stdout reader: <Thread(Thread-47 (_read_data), initial daemon)>
DEBUG gnupg      Thread-46 (_read_response)  985 [GNUPG:] KEY_CONSIDERED 9E26CD705E8C31F39A0E7E1C9BFFF293021B7D5E 0
DEBUG gnupg      Thread-46 (_read_response)  985 [GNUPG:] KEY_CONSIDERED 99BFC3DB10FE4BE821EA1228EF574800D803274C 0
DEBUG gnupg      Thread-46 (_read_response)  985 gpg: 8A3A613F82AAD7CD: There is no assurance this key belongs to the named user
DEBUG gnupg      Thread-46 (_read_response)  985 [GNUPG:] INV_RECP 10 99BFC3DB10FE4BE821EA1228EF574800D803274C
DEBUG gnupg      Thread-46 (_read_response)  985 [GNUPG:] FAILURE encrypt 53
DEBUG gnupg      Thread-46 (_read_response)  985 gpg: [stdin]: encryption failed: Unusable public key
WARNING gnupg      MainThread 1046 gpg returned a non-zero error code: 2

Failure 2:

DEBUG gnupg      MainThread  967 27422: gpg2 --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir /tmp/keys-2819fmm4 --debug-quick-random --encrypt --recipient F3C987C36C5C6343C9A5D5A1A3F494F6028E4866 --recipient FB61B9109C3DE42A98E9A3C2877E2F37005A7B82 --armor
DEBUG gnupg      MainThread  173 data copier: <Thread(Thread-92 (_copy_data), initial daemon)>, <_io.BytesIO object at 0x7f9d27cd0860>, <_io.BufferedWriter name=5>
DEBUG gnupg      MainThread 1029 stderr reader: <Thread(Thread-93 (_read_response), initial daemon)>
DEBUG gnupg      Thread-92 (_copy_data)  168 closed output, 12 bytes sent
DEBUG gnupg      MainThread 1036 stdout reader: <Thread(Thread-94 (_read_data), initial daemon)>
DEBUG gnupg      Thread-93 (_read_response)  985 [GNUPG:] KEY_CONSIDERED FB61B9109C3DE42A98E9A3C2877E2F37005A7B82 2
DEBUG gnupg      Thread-93 (_read_response)  985 gpg: checking the trustdb
DEBUG gnupg      Thread-93 (_read_response)  985 [GNUPG:] KEY_CONSIDERED FB61B9109C3DE42A98E9A3C2877E2F37005A7B82 0
DEBUG gnupg      Thread-93 (_read_response)  985 gpg: marginals needed: 3  completes needed: 1  trust model: pgp
DEBUG gnupg      Thread-93 (_read_response)  985 gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
DEBUG gnupg      Thread-93 (_read_response)  985 [GNUPG:] KEY_CONSIDERED F3C987C36C5C6343C9A5D5A1A3F494F6028E4866 2
DEBUG gnupg      Thread-93 (_read_response)  985 gpg: A3F494F6028E4866: There is no assurance this key belongs to the named user
DEBUG gnupg      Thread-93 (_read_response)  985 [GNUPG:] INV_RECP 10 F3C987C36C5C6343C9A5D5A1A3F494F6028E4866
DEBUG gnupg      Thread-93 (_read_response)  985 [GNUPG:] FAILURE encrypt 53
DEBUG gnupg      Thread-93 (_read_response)  985 gpg: [stdin]: encryption failed: Unusable public key
WARNING gnupg      MainThread 1046 gpg returned a non-zero error code: 2

Failure 3:

DEBUG gnupg      MainThread  967 27924: gpg2 --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir /tmp/keys-9z49t40e --debug-quick-random --verify
DEBUG gnupg      MainThread  173 data copier: <Thread(Thread-320 (_copy_data), initial daemon)>, <_io.BytesIO object at 0x7f9d27c5a160>, <_io.BufferedWriter name=5>
DEBUG gnupg      MainThread 1029 stderr reader: <Thread(Thread-321 (_read_response), initial daemon)>
DEBUG gnupg      Thread-320 (_copy_data)  168 closed output, 256 bytes sent
DEBUG gnupg      MainThread 1036 stdout reader: <Thread(Thread-322 (_read_data), initial daemon)>
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] NEWSIG
DEBUG gnupg      Thread-321 (_read_response)  378 message ignored: NEWSIG,
DEBUG gnupg      Thread-321 (_read_response)  985 gpg: Signature made Sun 19 Dec 2021 12:29:05 GMT
DEBUG gnupg      Thread-321 (_read_response)  985 gpg:                using DSA key FA9FB0DFE0431A24BF6942AFFA493F5794378D15
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] KEY_CONSIDERED FA9FB0DFE0431A24BF6942AFFA493F5794378D15 0
DEBUG gnupg      Thread-321 (_read_response)  378 message ignored: KEY_CONSIDERED, FA9FB0DFE0431A24BF6942AFFA493F5794378D15 0
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] SIG_ID naHfhoqnNFwY4NFn3LuKeuZHFXY 2021-12-19 1639916945
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] KEY_CONSIDERED FA9FB0DFE0431A24BF6942AFFA493F5794378D15 0
DEBUG gnupg      Thread-321 (_read_response)  378 message ignored: KEY_CONSIDERED, FA9FB0DFE0431A24BF6942AFFA493F5794378D15 0
DEBUG gnupg      Thread-321 (_read_response)  985 gpg: checking the trustdb
DEBUG gnupg      Thread-321 (_read_response)  985 gpg: no ultimately trusted keys found
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] GOODSIG FA493F5794378D15 Andrew Able (A test user (insecure!)) <andrew.able@alpha.com>
DEBUG gnupg      Thread-321 (_read_response)  985 gpg: Good signature from "Andrew Able (A test user (insecure!)) <andrew.able@alpha.com>" [unknown]
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] VALIDSIG FA9FB0DFE0431A24BF6942AFFA493F5794378D15 2021-12-19 1639916945 0 4 0 17 2 01 FA9FB0DFE0431A24BF6942AFFA493F5794378D15
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] KEY_CONSIDERED FA9FB0DFE0431A24BF6942AFFA493F5794378D15 0
DEBUG gnupg      Thread-321 (_read_response)  378 message ignored: KEY_CONSIDERED, FA9FB0DFE0431A24BF6942AFFA493F5794378D15 0
DEBUG gnupg      Thread-321 (_read_response)  985 [GNUPG:] TRUST_UNDEFINED 0 pgp
DEBUG gnupg      Thread-321 (_read_response)  985 gpg: WARNING: This key is not certified with a trusted signature!
DEBUG gnupg      Thread-321 (_read_response)  985 gpg:          There is no indication that the signature belongs to the owner.
DEBUG gnupg      Thread-321 (_read_response)  985 Primary key fingerprint: FA9F B0DF E043 1A24 BF69  42AF FA49 3F57 9437 8D15

Note: all keys are transient, created in the tests. You should be able to reproduce these results by cloning https://github.com/vsajip/python-gnupg and then running

GPGBINARY=/path/to/gpg-2.2.33 python3 test_gnupg.py and looking at the created test_gnupg.log.

Details

Version: 2.2.33

Revisions and Commits

rG GnuPG
	rG4cc724639c01 gpg: Fix adding the list of ultimate trusted keys.
	rG4aeeaa65ad09 gpg: Fix adding the list of ultimate trusted keys.
	rG610528725290 gpg: Correctly set the ownertrust for a new key.

Related Objects

Mentioned In: T5743: Release GnuPG 2.3.5
T5703: Release GnuPG 2.2.34
T5654: Release GnuPG 2.3.4
T5685: Clear stale --trusted-key records from the trustdb
Mentioned Here: rGbc6d56282ec9: gpg: Remove stale ultimately trusted keys from the trustdb.
rG5c37fd90bf81: * trustdb.h, trustdb.c (register_trusted_keyid): New. Adds a keyid to the
T5685: Clear stale --trusted-key records from the trustdb

Event Timeline

vsajip created this task.Dec 19 2021, 2:33 PM

vsajip created this object in space S1 Public.

vsajip renamed this task from Apparent regressions between 2.2.30 and 2.2.33 of GnuPG to Apparent regressions between 2.2.32 and 2.2.33 of GnuPG.Dec 19 2021, 3:18 PM

Please be so kind and describe the regressions you see. 3 log files from your software are not very helpful.

Okay, sorry. In the first two cases (encryption), GnuPG 2.2.33 generates

[GNUPG:] INV_RECP 10 F3C987C36C5C6343C9A5D5A1A3F494F6028E4866
[GNUPG:] FAILURE encrypt 53
gpg: [stdin]: encryption failed: Unusable public key

and exits with error code 2, whereas 2.2.32 doesn't display these messages and exits with return code 0.

In the third case (signature verification), GnuPG 2.2.33 generates

[GNUPG:] TRUST_UNDEFINED 0 pgp
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.

whereas 2.2.32 displays TRUST_ULTIMATE 0 pgp and verifies successfully, as it should.

vsajip updated the task description. (Show Details)Dec 19 2021, 8:04 PM

I think that the change for T5685 introduced the issue.

Symptom: When new key is created, trusted keys are cleared (except new one).

It should distinguish newly created keys and keys specified by --trusted-keys.

Here is a patch to describe the issue concretely:

gnupg23-trustdb-trusted-keys.patch7 KBDownload

The use of register_trusted_key in do_generate_keypair was a dirty hack utilizing a bug in --trusted-key ; it would be better to set the key as ultimately trusted.

So, this is the patch. Note that this is for master.

diff --git a/g10/keygen.c b/g10/keygen.c
index 7f15027a2..a452ab6d6 100644
--- a/g10/keygen.c
+++ b/g10/keygen.c
@@ -5619,7 +5619,7 @@ do_generate_keypair (ctrl_t ctrl, struct para_data_s *para,
           pk = find_kbnode (pub_root, PKT_PUBLIC_KEY)->pkt->pkt.public_key;
 
           hexfingerprint (pk, hexfpr, sizeof hexfpr);
-          register_trusted_key (hexfpr);
+          update_ownertrust (ctrl, pk, TRUST_ULTIMATE);
 
           if (!opt.flags.no_auto_trust_new_key)
             update_ownertrust (ctrl, pk,

We can even remove the hexfingerrprint call. Will go into 2.3.4. Thanks.

• werner added a commit: rG610528725290: gpg: Correctly set the ownertrust for a new key..Dec 20 2021, 11:03 PM

• werner mentioned this in T5654: Release GnuPG 2.3.4.Dec 20 2021, 11:08 PM

• gniibe edited projects, added gnupg (gpg22); removed gnupg.Dec 21 2021, 1:06 AM

Things are not that easy. I actually introduced a bug in 2.3.4. Here is a comment from my working copy:

/* On 2003-10-31 register_trusted_keyid was introduced and
  * called here: "Use it here so that the ultimate ownertrust
  * happens before the trustdb (might be) rebuilt.  Also fix
  * an error where the newly generated pk is thought to be a
  * subkey by the trustdb."
  *
  * On 2021-12-20 the register_trusted_keyid was removed and
  * replaced by update_ownertrust.  Obviously that defeats
  * the purpose of --no-auto-trust-new-key and the open
  * question is why we used to update the owenertruststead of
  * just setting it.
  */

 update_ownertrust (ctrl, pk, TRUST_ULTIMATE);

 if (!opt.flags.no_auto_trust_new_key)
   update_ownertrust (ctrl, pk,
                      ((get_ownertrust (ctrl, pk) & ~TRUST_MASK)
                       | TRUST_ULTIMATE ));

Will go into 2.3.4.

What about 2.2.34? Will the fix be backported?

Let me clarify:

It was on 2003-11-01 (ChangeLog is on 2003-10-31 probably in US): rG5c37fd90bf81: * trustdb.h, trustdb.c (register_trusted_keyid): New. Adds a keyid to the

• gniibe claimed this task.Jan 12 2022, 2:32 AM

Things have been a bit buggy here (probably, since the beginning).
In g10/trustdb.c,

functions accessing trustdb calls init_trustdb.
init_trustdb is executed only once, protected by trustdb_args.init variable.
from init_trustdb, verify_own_keys is called, which updates utk_list from user_utk_list
tdb_register_trusted_keyid or tdb_register_trusted_key adds keyid to user_utk_list

This mechanism works only when tdb_register_trusted_key/id is called before any access to trustdb, at some initialization like by --trusted-key option. Update at keygen would work, but only for a single generation. (Second time, it won't work.)

For keygen and editing ownertrust, I think that we need to expose a function from trustdb.c which updates utk_list.

In the original code, register_trusted_keyid is used in keygen.c, so that it updates user_utk_list, thus, will be into utk_list.
This should be done, by adding the keyid to utk_list directly.

• gniibe added a commit: rG4aeeaa65ad09: gpg: Fix adding the list of ultimate trusted keys..Jan 12 2022, 5:42 AM

Here is the backport to 2.2:

gnupg22-utk-change.diff3 KBDownload

no call from keygen.c to register_trusted_keyid, as the semantics has been changed with --trusted-key by the commit rGbc6d56282ec9: gpg: Remove stale ultimately trusted keys from the trustdb..
remove register_trusted_keyid function.
Add call to tdb_update_utk in update_onwertrust, so that keyid is added to the list directly (not through user_utk_list).
New function tdb_update_utk

Thanks for diving into the history of that code.

Backported to 2.2, too.

• gniibe added a commit: rG4cc724639c01: gpg: Fix adding the list of ultimate trusted keys..Jan 17 2022, 6:35 AM

• werner mentioned this in T5703: Release GnuPG 2.2.34.Feb 7 2022, 10:06 PM

• gniibe closed this task as Resolved.Feb 17 2022, 5:46 AM

• werner mentioned this in T5743: Release GnuPG 2.3.5.Apr 21 2022, 5:59 PM