Closing, given that we implemented a general solution; see the parent task.

I have seen no responses on your two mails to the ML and given th athere is no concrete protocol bug, I close this issue. If you can show a concrete bug please re-open this issue again.

Apparently, it's an error from your installed /usr/local/opt/libgpg-error/lib/libgpg-error.0.dylib (you have some configuration to prefer this library), while your configure is for /usr/local/lib (because you specify no --prefix).

In FreeBSD, getrandom(3) became available, when getrandom(2) was added. <-- This is my theory.
If this is true, just use getrandom(3), not using getrandom(2) by syscall.

Adding the patch here.

T3530 describes the solution. In short: Put "auto-expand-secmem" into gpg-agent.conf.

Thanks. Just pushed the change to master.

The corresponding fix can be found here: https://github.com/smuellerDD/jitterentropy-library/commit/9048af7f06fc1488904f54852e0a2f8da45a4745

Please note that this issue in Jitterentropy has been already fixed by upstream: http://www.chronox.de/jent.html

Hi sorry, here it is. I don't see a recommended way for providing a ton of text, so just pasting it here.

Please provide a complete build log or at least the output of the configure run.

IIUC, in Gentoo multilib (or other distributions), <triplent>-{gpg-error,libgcrypt,libassuan,npth,libksba,npth}-config script is used.
In forthcoming libgpg-error 1.33, single gpgrt-config is used for all architecture, by having --libdir option at invocation time.

New gpg-error.m4 detects gpgrt-config, too.
And configure supplies --libdir when it invokes gpgrt-config.
For other *.m4 (libassuan, ksba, libgcrypt, ntbtls), it is possible for them to check GPGRT_CONFIG to use gpgrt-config if any.
For npth.m4, it can do that too, with no hard dependency to libgpg-error.

Fixed in master and 1.8.
@dkg: Thanks for the comments and your patience to convince me.

The next step is to release libgcrypt 1.8.4 :-)

Actually we plan to provide a more convenient way to perform the DH operation. See for example P7 for the non-elegant way which is required today.

Fixed in master and 1.8 by detecting a fork and re-opening the devices

A bit tricky, but this would be good to use gpgrt-config by gpg-error.m4.
I say "tricky", because its name is gpg-error.m4 but it configure GPGRT_CONFIG to access to GPG_ERROR_CONFIG.

It might be good idea to provide libgcrypt.pc in libgcrypt 1.8.x for forward compatibility with libgpg-error 1.33.

Well, I changed my mind. Use of new gpgrt-config requires software update to introduce gpgrt.m4 and update of configure.ac to switch gpgrt from gpg-error, in standard way.
That's too much this time. It's good to defer this change.

OK, I'll change to use gpgrt-config, along with requiring newer version of libgpg-error.

Fixed in 1.8

Fixed in 1.8

yat2m updated. Thanks.

Thanks.

Thanks again.

Thanks. Not a real world problem now but needs to be fixed.

May I suggest to use a (new) gpgrt-config instead of the current name libgpg-error-config. The long term plan is to change the name of the library.

This is the dependency graph:

Thanks. Fixed in master. Needs backport.

Thanks. Fixed in master.

Done for libgcrypt, finally.

ntbtls adopts the way from the beginning.

Please check your ld.so (dynamic linker) setting (/etc/ld.so.conf and/or LD_LIBRARY_PATH).

Done for gpgme.

What are the next steps here? i confess i'm a little tired of doing regular checkins on this issue, and i'm sure other people are tired of me nagging. What can we do to move this along?

secmem routines are installed into gniibe/secmem branch.
Please note that it's only secmem routines, not malloc_secure.

Thanks for your comments, Stephan.

@aheinecke -- @smueller_chronox.de (author of the comment above) is Stephan Müller from atsec. Glad to see he seems ok with the proposal :)

Apologies for not having read all comments in this long thread. I was asked to comment on the patch, so here is my comment:

I created gniibe/secmem branch for this.
https://dev.gnupg.org/source/libgpg-error/history/gniibe%252Fsecmem/

well, i tried to push, anyway, but it looks like playfair is rejecting my pushes:

@werner -- yes, i am asking for a change that is specific to the way that gcrypt interacts with the Linux kernel. The minor patch i've proposed only affects a codeblock within #if defined(__linux__), so i don't believe it would have an effect on other Unices. I hope that people working with other kernels will propose any necessary fixes for them.

@kallisti5: For you server you can add only_urandom to random.conf when changing to a multiuser runlevel and remove it early at startup and termination.

/dev/random, RDRAND, etc involves a lot of political arguments and thus it is not easy to decide what to do. What you are calling for is a linux kernel specific code path (note that rndlinux is used by most Unices) and won't be helpful for other OSes. I am of course willing to do add specific for for a few widespread OSes (and in any case for Debian). It is a major change and thus does not belong into 1.8 - I am fine with master which Debian might want to backport.

Well, Werner is just back so he can say more.
An excellent reviewer was Stephan Müller from atsec. He is also involved a bit afaik in the kernel random development.

@aheinecke thanks for the followup!

We are moving to use the yat2m from gpgrt (libgpg-error); thus the additional tag.

I think that the ultimate decision here lies with Werner. Additional review.
I think the biggest obstacle is that we don't want to change the random gathering code if it can be avoided and that the random code has been thoroughly reviewed / tested and is currently considered secure.

This bug report has been around for several months now. it has a simple patch, a clear explanation, a report of running code, and examples of problems it solves.

Good to know, no problem, just wanted to document it just in case they do remove the API entirely in the future.

CryptGenRandom is only used as an additional source of entropy and doesn't count towards our entropy estimation. Thus whether it is used of not does not make any difference. Our main entropy source is meanwhile the jitter based RNG. Thus your request will receive a low priority.

I've now run the proposed patch on a GNU/Linux system where the kernel's RNG is initialized but /proc/sys/kernel/random/entropy_avail shows numbers below 100, and i can confirm that 3072-bit RSA key generation takes roughly 0.8 seconds: 20 sequential default --quick-keygen operations (each creating two secret keys) took ~32s.

Here is another example of users doing sketchy things to try to "fix" this process:

Here is an example of the kinds of UI/UX mystery that users face while this decision is unresolved:

Done for npth.

User input, anything to solve the lack of entropy on servers would be *great*. We have a bunch of buildbot workers we would *love* to have sign their artifacts... however we end up (unsuccessfully) doing stupid things like this to try and drive up entropy as a non-root user:

Looking at the table in random(7) it seems clear to me that what we want to just invoke getrandom() with no arguments. This blocks until the kernel's PRNG has been adequately seeded, but once seeded it doesn't block, while still pulling from an unbreakably-strong PRNG. this is the best-of-both-worlds situation that we want.

Changing the GnuPG long-term (and short-term) key generation techniques to use this approach might require coordination with gcrypt. gcrypt's gcry_random_level currently has GCRY_WEAK_RANDOM and GCRY_STRONG_RANDOM and GCRY_VERY_STRONG_RANDOM, which doesn't represent the nuance described above.

One approach might be to just have gcrypt on Linux treat all values of gcry_random_level the same, and use getrandom() for all of them.

ping again…

could i get feedback on this ticket? a simple, clean patch is available, and i don't understand what is blocking it.

Thanks.
So what I remembered was 1 year and 1 month off the real EOL date.