fwiw, i've just shipped a patch to correct this change in behavior in the 2.2 branch debian. Many thanks to @gniibe , on whose work in the 2.4 branch this is based, and to @ametzler1, who did the backporting to 2.2. I've also written a test which tries to tickle this bug. It fails with unpatched 2.2.43 as emacs times out signing and encrypting mail as epg.el deadlocks with gpg.

This is not a bug. We changed it as a convenience for some Emacs users.

Are you saying that concern about "risking a regression" is the reason to not fix this bug, which is itself a regression, and was introduced into the a point release in the current "long term support" branch?

Sorry, no. The change is too large to back port it w/o risking a regression. As mentioned in T6481#170366 I don't consider this a bug. We are anyway working towards version 2.6 which will be the next LTS version.

Any chance this could also be fixed in the 2.2.x series, where it seems to have been introduced in 2.2.42?

I upgraded to gnupg 1.4.4 now, the problem is gone. Thanks for working.

Thanks @gniibe and everybody!

Fixed in GnuPG 2.4.4.

In T6481#181779, @gniibe wrote:

Arch Linux: https://gitlab.archlinux.org/archlinux/packaging/packages/gnupg
FreeBSD: https://cgit.freebsd.org/ports/tree/security/gnupg

I don't see the patch is applied. Please wait for GnuPG release 2.4.4.

Indeed, openSUSE has applied the patch: https://build.opensuse.org/package/show/openSUSE%3AFactory/gpg2

Arch Linux: https://gitlab.archlinux.org/archlinux/packaging/packages/gnupg
FreeBSD: https://cgit.freebsd.org/ports/tree/security/gnupg

Works as expected on openSUSE Tumbleweed with gpg2-2.4.3-4.2.x86_64:

$ gpg2 --version
gpg (GnuPG) 2.4.3
libgcrypt 1.10.3
[...]

In T6481#181724, @gniibe wrote:

i still observe the same behavior:

What do you mean? I can't replicate the behavior described by you, using the GnuPG from the repo, or the one of Debian 2.4.3-2.

i still observe the same behavior:

In T6481#171399, @gniibe wrote:

• It is also possible for GnuPG to keep the behavior of 2.4.0, so that it doesn't confuse EasyPG.
  • This is a possible solution #2: change in master: rG2f872fa68c65: gpg: Report BEGIN_* status before examining the input.
  • But... it is difficult for implementation of GnuPG to guarantee this kind of behavior.

For a while, distributions can apply rG2f872fa68c65 for 2.4 series.

Applied to 2.4, too.

ack

@werner: What do you mean by "We have a fix for now"? Are you referring to @gniibe's patch?

We have a fix for now and thus I lower the priority. Given that EasyPG mimics the GPGME API we should here also use another pipe to convey the passphrase (e.g. for symmetric encryption).

I consider the entire idea of receiving a passphrase and data on the same channel to be a bad for security and robust coding. The whole thing is a historical oddity which we kept for the sake of mutt(1)'s legacy way of invoking pgp. Thus I won't consider 3) the best option.

To summarize, here is the situation:

• Ideally, it would be good to modify GnuPG and Emacs EasyPG to implement status handling and input handling in better way.

Ah, I see https://dev.gnupg.org/rG2f872fa68c6576724b9dabee9fb0844266f55d0d applies cleanly. I guess can go with that, although would prefer it if on the 2.4 branch.

Is there a commit we could backport downstream to 2.4.x? We've had quite a few reports of this.

I pushed the change which keeps old status report behavior to master.
Let me test the change.

looks simpler to me.

Hmm, for the latter this:

Orthogonally, here is possible change for GnuPG, if we need to support the workaround of compress-level 0 in ~/.gnupg/gpg.conf.

OK, here is my changes which always use make-temp-file (to avoid confusion between data input and passphrase input).

I use epg.el with the change of removing the wait:

@werner We could make the wait conditional on (equal epg-gpg-program "gpg"), that is, only when user has GnuPG 1.x.

Well okay, then I have no workaround. However, I won't consider this a bug because BEGIN_ENCRYPTION marks the start of the actual encryption process but not when it starts to read input data.

The change rG60963d98cfd8: gpg: Detect already compressed data also when using a pipe. for T6332 introduce IOBUF_IOCTL_PEEK.

@werner I tested by switch back to GnuPG 2.4.1 (I downgraded to 2.4.0 before to temporary work around issue), adding compress-level 0 to gpg.conf file. It's not working. The problem still exist.

I have not yet experienced that although I am using Gnus with encrypted mail all the time. My guess is that this is due to the improved compressed input detection in gpg. You might be able to work around it by adding compress-level 0 to gpg.conf

pinentry-emacs is obsolete. It's for older Emacs (<= 25, IIUC) which had lisp/pinentry.el.
For Emacs 26 and newer, you can simply use epa-pinentry-mode having the value of loopback.

That's a fair point, cheers!

Let's turn this into a feature request.

No worries -- you led me in the direction of a solution when you mentioned loopback mode. I appreciate your time and your help!

Sorry, I responded in a mode of "tracking a bug to fix soonish". I should have changed my mode into showing HOWTO.
Thanks for sharing useful link.