Problem with SHA-256 on x86-64 is that it took long time for Intel to introduce SHA acceleration (SHA1 & SHA256) to their main CPU products.

Fixed in master.
Will backport into 2.4, after testing.

So some more data from 2.4.3. Once unpatched and once with your patch.

Am I correct that the reason for the speed up is that it can use a second CPU's engine. If there is a real performance improvement here, we should add this for example using a --compatibility-flag. This way we can gather experience and eventually make it the default. The compatibility flags won't introduce an API incompatibility.

I found this important to do because the SHA-256 results from your test looked extremly slow.

For comparison here are my benchmark results on the same system. Once on Linux 64 bit and once on W32 and once with W64. All runs are native to exclude any virtualization issues.

Updated the patch:

I did a little bit of testing with okular, and it kind of gives similar numbers.

This is a patch for master (which uses nPth for gpg):

I'm trying to use a thread for computation of hashing. It works, but no performance gain, due to nPth.
I do:

npth_unprotect ();
gcry_md_write (mfx->md, mfx->buf, mfx->written);
npth_protect ();

to expect hashing is done in parallel.

It seemed I was wrong that it is due to buffering.
In the use case of --sign and --encrypt, hashing is done with IOBUF's 64KiB buffer (already).

I observed the benchmark by libgcrypt (Windows emulation 32-bit on Debian):

OK. I'll take the signing part (possible performance improvement).

I looked through the code. What I observed is:

• By jussi's improvements, AEAD code is optimized with AEAD_ENC_BUFFER_SIZE of 64KiB
  • this contributes much for better performance
• If we invoke gpg --sign | gpg --encrypt then we can take advantage of multiple CPUs (but gpg is currently not automatically threaded in that way)
  • signing could be improved likewise, using larger buffer like 64KiB
• CFB+MDC, it uses two functions together; encryption and hashing, and not with larger buffer like 64KiB
  • when signed, it also does hashing for signing, so three functions

! In T6561#172087, @werner wrote:
FWIW, gpg shows the actual cipher and encryption mode with -v. For example

FWIW, gpg shows the actual cipher and encryption mode with -v. For example

In T6561#172083, @werner wrote:

s/CBC/CFB+MDC/

s/CBC/CFB+MDC/

See for T6545 for a new request to support IDP.

Thank you.
Applied to master, 2.4 branch and 2.2 branch.

Thank you.
Applied to master, 2.4 branch, and 2.2 branch.

I consider the entire idea of receiving a passphrase and data on the same channel to be a bad for security and robust coding. The whole thing is a historical oddity which we kept for the sake of mutt(1)'s legacy way of invoking pgp. Thus I won't consider 3) the best option.

To summarize, here is the situation:

• Ideally, it would be good to modify GnuPG and Emacs EasyPG to implement status handling and input handling in better way.

Ah, I see https://dev.gnupg.org/rG2f872fa68c6576724b9dabee9fb0844266f55d0d applies cleanly. I guess can go with that, although would prefer it if on the 2.4 branch.

Is there a commit we could backport downstream to 2.4.x? We've had quite a few reports of this.

To align the default expiration time with the BSI approval and other related software we change this now to 3 years.

The problem of hang of tests/openpgp/multisig.scm is solved by rGef4f22b9d98b: gpg: Graceful exit for signature checking with --batch.
But the problem itself is not yet solved.

It is reproducible by testing tests/openpgp/multisig.scm with keyboxd enabled (it hangs), with the modification of following.

I pushed the change which keeps old status report behavior to master.
Let me test the change.

looks simpler to me.

Hmm, for the latter this:

Orthogonally, here is possible change for GnuPG, if we need to support the workaround of compress-level 0 in ~/.gnupg/gpg.conf.

OK, here is my changes which always use make-temp-file (to avoid confusion between data input and passphrase input).

I use epg.el with the change of removing the wait:

Was resolved, see T4457

GnuPG is and can't be FIPS-140-3 compliant due to the way it is implemented. We may eventually employ the new hash-and-sign API of Libgcrypt to move into this direction but that has not yet been done. However, this also requires the use of the new indicator API and the, well, a RedHat kernel.

Pushed to GnuPG master. Let us test. For my machine of Debian GNU/Linux, Wine emulation (Windows 32-bit, Windows 64-bit), make check goes all well.
After confirming the implementation, I'd like to put it into gpgrt.

Will be in 2.4.2

@werner We could make the wait conditional on (equal epg-gpg-program "gpg"), that is, only when user has GnuPG 1.x.

Well okay, then I have no workaround. However, I won't consider this a bug because BEGIN_ENCRYPTION marks the start of the actual encryption process but not when it starts to read input data.

The change rG60963d98cfd8: gpg: Detect already compressed data also when using a pipe. for T6332 introduce IOBUF_IOCTL_PEEK.

@werner I tested by switch back to GnuPG 2.4.1 (I downgraded to 2.4.0 before to temporary work around issue), adding compress-level 0 to gpg.conf file. It's not working. The problem still exist.

I have not yet experienced that although I am using Gnus with encrypted mail all the time. My guess is that this is due to the improved compressed input detection in gpg. You might be able to work around it by adding compress-level 0 to gpg.conf

Just checked 2.4.1 and looks like now everything is OK.

works, Gpg4win-4.1.1-beta295

Sorry, the comment above is my misunderstanding.

For importing key/subkey case, it doesn't matter if the smartcard is connected or not. The data in the file will be overwritten by import.

This works if the smartcard with the same private key is not connected, which it usually shouldn't be (outside of testing situations) so that's ok for me.
But I think we should inform the user what is done or isn't.
Currently we get in both cases:

gpg: Anzahl insgesamt bearbeiteter Schlüssel: 1
gpg:                             unverändert: 1
gpg:              gelesene geheime Schlüssel: 1
gpg:            geheime Schlüssel importiert: 1

Which is not very clear in any case but in case the smart card is connected it is in fact wrong, there is no secret key imported, the keystub remains.

Another miscellaneous correction for jimregexp. A condition was copy-pasted from another section without the necessary changes, resulting in incorrect logic. This seems harmless apart from inconsistent error reporting.

diff --git a/regexp/jimregexp.c b/regexp/jimregexp.c
index 1a8b8aae6..1b6e1b49c 100644
--- a/regexp/jimregexp.c
+++ b/regexp/jimregexp.c
@@ -778,7 +778,7 @@ static int regatom(regex_t *preg, int *flagp)
                                                        preg->err = REG_ERR_NULL_CHAR;
                                                        return 0;
                                                }
-                                               if (start == '\\' && *pattern == 0) {
+                                               if (end == '\\' && *pattern == 0) {
                                                        preg->err = REG_ERR_INVALID_ESCAPE;
                                                        return 0;
                                                }

Pushed the change not including OK_TO_CHANGE_ERROR_BEHAVIOR part.
Note that the modification affects main key case, too.

To minimize the impact of the change, I updated:

diff --git a/g10/import.c b/g10/import.c
index 1ed40a63c..345e8cc75 100644
--- a/g10/import.c
+++ b/g10/import.c
@@ -2955,9 +2955,23 @@ do_transfer (ctrl_t ctrl, kbnode_t keyblock, PKT_public_key *pk,
 {
   gpg_error_t err;
   struct import_stats_s subkey_stats = {0};
+  int force = 0;
+  int already_exist = agent_probe_secret_key (ctrl, pk);
+
+#ifndef OK_TO_CHANGE_ERROR_BEHAVIOR
+  if (already_exist == 1)
+    return gpg_error (GPG_ERR_EEXIST);
+#endif
+  if (already_exist == 2)
+    {
+      if (!opt.quiet)
+        log_info (_("key %s: card reference is overridden by key material\n"),
+                  keystr_from_pk (pk));
+      force = 1;
+    }

Changes may be something like:

diff --git a/g10/import.c b/g10/import.c
index 1ed40a63c..91ff0c8ec 100644
--- a/g10/import.c
+++ b/g10/import.c
@@ -2706,6 +2706,20 @@ transfer_secret_keys (ctrl_t ctrl, struct import_stats_s *stats,
           goto leave;
         }

isn't T3456 the same issue?

On gpg4win 4.1.0 (and GnuPG VSD 3.1.26) there are no longer password prompts for the subkeys when exporting (or making a backup from) secret keys.

This has been solved loooong ago.

OCB mode (i.e. packet 20) is only used if the keys announce it. Thus only after moving a (private) key from GnuPG to a non-GnuPG compatible implementation you will run into this problem. The compatibility options won't override the preference system.

gpg-agent now supports READKEY --card command which creates stub file when it's not yet available on host computer.
It was implemented by rG82cbab906a3e: agent: Add --card option for READKEY.

Thanks for the description; this is good for documentation.

That sounds like a great (and better) solution! Thank you for your attention & effort 👍🙏

See T6395 for the new feature. It will be released with 2.4.1 but it will take some time that it can actually be used because the other party needs to have an OpenPG implementation which supports this.

Please use

gpgtar -C /home/matt/data ....

instead of using an absolute name. This makes things much easier to implement in a secure way: You don't want to have absolute file names in the tarball and mapping them to relative names is not easy or even impossible in case of, say "/home/foo/x.data /home/bar/x.data". Keep in mind that gpgtar does also not handle symlinks and other special files.

I guess this is fixed with this commit for 2.2. and 2.4. Given that the report is quite old with not new infos since 2019, I'll close it.

The reason why gpg does not encrypt to multiple subkeys is that the older subkeys are viewed as deprecated. You could write a tool which does a heuristic to check when the time is reached that no more messages are encrypted to an older subkey (or are used to decrypt archived mails). At that point you can take the private part of the old subkey offline.

Debian's wiki also speaks a lot about the advantages & dream of subkeys, but also mentions the caveat:

I've read many articles mentioning the improved key handling when different devices just have different subkeys, thus allowing a semantic connection to a primary identity (instead of different "Identities" on different devices)

What do you want to achieve by using multiple encryption subkeys? Do you realize that gpg will always encrypt to one subkey (unless you explicitely specify multiple subkeys), i.e. you won't be able to decrypt on device 1 what you have encrypted for device 2 and vice-versa. Usually, this makes little sense because it seems you want to be able to decrypt anything on your main machine.

Thanks for the reply :)

Sorry, I think you have to fix the other tools. The ! suffix has virtually been supported forever and any new option to do the same complicates the code and the documentation.

Kleopatra simply copies the content of the corresponding *.key file in the private-keys-v1.d folder. If the *.key file contains a shadowed key after issuing a KEYTOCARD --force [...] command followed by a SCD LEARN --force command (note the SCD!), then gpg-agent is to blame.

Works now for gpgme. Thanks!

Understood. I appreciate the time you took to analyse the issue. Thanks.