Page MenuHome GnuPG
Create Task
Maniphest T8197

"gpg --refresh-keys" aborts with "gpg: keyserver refresh failed: No data" if too many keys are missing on keyserver
Closed, ResolvedPublic
Actions

Description

Affected: at least GnuPG versions 2.4.4 2.4.7(=Debian 13 stable) 2.4.8 2.5.18

Summary: if gpg is told to update its keys from a key server via the following command, the updates are aborted with an error in many cases:

gpg --keyserver-options no-self-sigs-only,no-import-clean --keyserver hkps://example.com/ --refresh-keys

We're using a DIY key server which follows the HKPS-specifications. We're pretty sure this is an issue at client side. Other GnuPG implementations are just working fine.

According to an analysis of https://github.com/gpg/gnupg, the issue is:

  • Keys are fetched in chunks. For 40-hex fingerprints, that is typically about 21 keys per chunk.
  • Within a chunk, keys are tried one by one.
  • If at least one key in that chunk is fetched successfully, that chunk counts as success, and GnuPG proceeds to the next chunk.
  • If a chunk returns no data at all, meaning none of the keys in that chunk could be fetched, keyserver_get stops there and the refresh aborts for all later chunks.

In detail:

  • g10/keyserver.c:1215 calls g10/keyserver.c:1598 for the batch.
  • g10/keyserver.c:1609 processes the batch chunk by chunk and stops on the first chunk error.
  • Inside one chunk, dirmngr's dirmngr/ks-action.c:361 fetches keys one by one, and a missing key does not immediately abort the others; it saves the error and continues.
  • But if that chunk produces no data overall, dirmngr returns GPG_ERR_NO_DATA at dirmngr/ks-action.c:445, and then keyserver_get aborts further chunks at g10/keyserver.c:1615.

So if this analysis holds true: gnupg fetches the keys in batches (chunks) of, e.g., 21 keys per batch. If all keys of a single batch of 21 keys do not have a representation on the key server, gnupg aborts the overall update process with an error, not fetching all the key updates from the server.

I've implemented a workaround script to mitigate the issue until the bug is fixed in all relevant gnupg versions: https://codeberg.org/publicvoit/my-domain-gpg-refresh-keys.sh

HTH

Details

Version
2.4.4 2.4.7 2.4.8 2.5.18

Revisions and Commits

rG GnuPG
rGfa1bbe348f3d gpg: Fix edge case in --refresh-keys
rGfd0d620a6e15 gpg: Fix edge case in --refresh-keys

Event Timeline

Karl created this task.Fri, Mar 27, 4:28 PM
• werner triaged this task as High priority.Sat, Mar 28, 6:12 PM
• werner added projects: gnupg26, Keyserver.
thesamesam added a subscriber: thesamesam.Mon, Mar 30, 10:42 AM
• werner added a subscriber: • werner.Tue, Mar 31, 3:18 PM

Can you please test the patch below in your environment. That would be helpful.

diff --git a/g10/keyserver.c b/g10/keyserver.c
index 99a5e350a..d1c050a95 100644
--- a/g10/keyserver.c
+++ b/g10/keyserver.c
@@ -1612,6 +1612,15 @@ keyserver_get (ctrl_t ctrl, KEYDB_SEARCH_DESC *desc, int ndesc,
                                  override_keyserver, flags, r_fpr, r_fprlen);
       if (!err)
         any_good = 1;
+      if (gpg_err_code (err) == GPG_ERR_NO_DATA)
+        {
+          /* That error is returned by dirmngr from its own loop.  We
+           * clear the error if more descriptions are available so
+           * that the next chunk gets retrieved.  In the other case
+           * the error will cause the termination of the loop.  */
+          if (ndesc_used < ndesc)
+            err = 0;
+        }
       if (err || ndesc_used >= ndesc)
         break; /* Error or all processed.  */
       /* Prepare for the next chunk.  */
• werner added a commit: rGfd0d620a6e15: gpg: Fix edge case in --refresh-keys.Tue, Apr 7, 4:46 PM
• werner moved this task from Backlog to WIP on the gnupg26 board.Tue, Apr 7, 4:46 PM
• werner added a project: gnupg22.

Applied to master to be release with 2.5.19.

Karl added a comment.Wed, Apr 8, 4:36 PM

@werner I can confirm that we've tested the patch and it seems to fix the issue in our setup.

• werner added a commit: rGfa1bbe348f3d: gpg: Fix edge case in --refresh-keys.Thu, Apr 16, 3:12 PM
• werner closed this task as Resolved.Thu, Apr 16, 3:13 PM
• werner claimed this task.
• werner moved this task from Backlog to QA on the gnupg22 board.

Reporter has tested 2.5 - the code in 2.2 is identical; no need for separate testing

• werner moved this task from WIP to Done on the gnupg26 board.Thu, Apr 16, 3:14 PM
• werner moved this task from QA to gnupg-2.2.54 on the gnupg22 board.
• werner edited projects, added gnupg22 (gnupg-2.2.54); removed gnupg22.