Done for libgcrypt, finally.

ntbtls adopts the way from the beginning.

Please check your ld.so (dynamic linker) setting (/etc/ld.so.conf and/or LD_LIBRARY_PATH).

Done for gpgme.

What are the next steps here? i confess i'm a little tired of doing regular checkins on this issue, and i'm sure other people are tired of me nagging. What can we do to move this along?

secmem routines are installed into gniibe/secmem branch.
Please note that it's only secmem routines, not malloc_secure.

Thanks for your comments, Stephan.

@aheinecke -- @smueller_chronox.de (author of the comment above) is Stephan Müller from atsec. Glad to see he seems ok with the proposal :)

Apologies for not having read all comments in this long thread. I was asked to comment on the patch, so here is my comment:

I created gniibe/secmem branch for this.
https://dev.gnupg.org/source/libgpg-error/history/gniibe%252Fsecmem/

well, i tried to push, anyway, but it looks like playfair is rejecting my pushes:

@werner -- yes, i am asking for a change that is specific to the way that gcrypt interacts with the Linux kernel. The minor patch i've proposed only affects a codeblock within #if defined(__linux__), so i don't believe it would have an effect on other Unices. I hope that people working with other kernels will propose any necessary fixes for them.

@kallisti5: For you server you can add only_urandom to random.conf when changing to a multiuser runlevel and remove it early at startup and termination.

/dev/random, RDRAND, etc involves a lot of political arguments and thus it is not easy to decide what to do. What you are calling for is a linux kernel specific code path (note that rndlinux is used by most Unices) and won't be helpful for other OSes. I am of course willing to do add specific for for a few widespread OSes (and in any case for Debian). It is a major change and thus does not belong into 1.8 - I am fine with master which Debian might want to backport.

Well, Werner is just back so he can say more.
An excellent reviewer was Stephan Müller from atsec. He is also involved a bit afaik in the kernel random development.

@aheinecke thanks for the followup!

We are moving to use the yat2m from gpgrt (libgpg-error); thus the additional tag.

I think that the ultimate decision here lies with Werner. Additional review.
I think the biggest obstacle is that we don't want to change the random gathering code if it can be avoided and that the random code has been thoroughly reviewed / tested and is currently considered secure.

This bug report has been around for several months now. it has a simple patch, a clear explanation, a report of running code, and examples of problems it solves.

Good to know, no problem, just wanted to document it just in case they do remove the API entirely in the future.

CryptGenRandom is only used as an additional source of entropy and doesn't count towards our entropy estimation. Thus whether it is used of not does not make any difference. Our main entropy source is meanwhile the jitter based RNG. Thus your request will receive a low priority.

I've now run the proposed patch on a GNU/Linux system where the kernel's RNG is initialized but /proc/sys/kernel/random/entropy_avail shows numbers below 100, and i can confirm that 3072-bit RSA key generation takes roughly 0.8 seconds: 20 sequential default --quick-keygen operations (each creating two secret keys) took ~32s.

Here is another example of users doing sketchy things to try to "fix" this process:

Here is an example of the kinds of UI/UX mystery that users face while this decision is unresolved:

Done for npth.

User input, anything to solve the lack of entropy on servers would be *great*. We have a bunch of buildbot workers we would *love* to have sign their artifacts... however we end up (unsuccessfully) doing stupid things like this to try and drive up entropy as a non-root user:

Looking at the table in random(7) it seems clear to me that what we want to just invoke getrandom() with no arguments. This blocks until the kernel's PRNG has been adequately seeded, but once seeded it doesn't block, while still pulling from an unbreakably-strong PRNG. this is the best-of-both-worlds situation that we want.

Changing the GnuPG long-term (and short-term) key generation techniques to use this approach might require coordination with gcrypt. gcrypt's gcry_random_level currently has GCRY_WEAK_RANDOM and GCRY_STRONG_RANDOM and GCRY_VERY_STRONG_RANDOM, which doesn't represent the nuance described above.

One approach might be to just have gcrypt on Linux treat all values of gcry_random_level the same, and use getrandom() for all of them.

ping again…

could i get feedback on this ticket? a simple, clean patch is available, and i don't understand what is blocking it.

Thanks.
So what I remembered was 1 year and 1 month off the real EOL date.

Here is our announcement: https://lists.gnupg.org/pipermail/gnupg-announce/2018q2/000426.html

https://www.nccgroup.trust/us/our-research/technical-advisory-return-of-the-hidden-number-problem/

Informed Debian security team about our change of libgcrypt.

A new installer for GnuPG with Libgcrypt 1.8.3 is now available.

Releases are now available. Next task is to build a new GnuPG Windows installer.

1.8.3 and 1.7.10 are now released. Announcement will follow later the day.

Pushed fixes to the repository at 16:00+0900 (09:00+0200). It's 0700Z.

In master, it's

commit 9010d1576e278a4274ad3f4aa15776c28f6ba965
Author: NIIBE Yutaka <gniibe@fsij.org>
Date:   Wed Jun 13 15:28:58 2018 +0900

1.8.3 has not yet been released and thus there is no NEWS entries and there can't be a 1.8.3 tag. You are right that the README still says 1.7. I'll fix that for 1.8.3. Why do you think maintenance of 1.7 stopped; the AUTHORS file and the new EOL statements on the download page say that we are going to maintain it until 2019-06-30.

Publication is planned for the 13th, 1500Z

I just noticed, that a tag for Libgcrypt 1.8.3 seems to be missing: https://dev.gnupg.org/source/libgcrypt/tags/LIBGCRYPT-1.8-BRANCH/

I've heard no critique of the logic above. could we get this fix landed? it is concretely useful for doing key generation on modern GNU/Linux systems.

I changed the priority to 'Normal'. The problem now is not the libssh usage, but how we can assume use of secure memory by random generator(s).

By libssh upstream, the problem has been fixed: commit-72f6b34

Here is the function:
https://git.libssh.org/projects/libssh.git/tree/src/dh.c#n227

Am I right to assume that the test suite is terminating and restarting libgcrypt? Although we have features for this, I am still not convinced that this is a proper use of libgcrypt. There are just too many cases how this can fail. Unix is not designed to use shared libraries in so-called "plugins". I need to look closer at the libssh code.

It would be better not to require gcry_control(GCRYCTL_CLOSE_RANDOM_DEVICE). Automatic handling through gcry_control(GCRYCTL_TERM_SECMEM) would be better.

The patch D461 makes gcry_control(GCRYCTL_CLOSE_RANDOM_DEVICE) free the allocated secure memory.

It assumes a change of libssh like:

Here is my patch: D461: jent random requires finalizer to deallocate secure memory

I backported the fix for 1.8.3.

Cherry-picked this for 1.8.3.

FIPS rules changed anyway and thus more rework will be needed anyway. I keep this open at low priorirty.

Thank you :)

Thanks. I only now noticed that this is the same as we already use for 32 bit MIPS. I have no more questions. Will push to master and the 1.8 branch.

Clang doesn't support the "h" inline asm constraint and the C version of umul_ppmm() works on MIPS64.

Your patch indicates that all clang versions for MIPS64 support this feature. Is my reading correct?

Got the question about this note from a user (in a internal email) and I see the problem that users do not have enough information to decide this. They do not know what the consequences of this note are (and suspect it to be the cause of error of they see it together with other problems). So to me it is more than a 'wish' as it will generate questions and leaves users in a situation where they cannot progress by their own in most of the situations.

It is not an error or even a warning but just a NOTE. Thus the user should decide. it is not even translated and most systems this is enabled anyway.

See also Filippo Valsorda's 32c3 talk about CSPRNGs.

@gouttegd : setting only-urandom at the distro level problematic due to two factors:

@dkg : Can’t this be solved at the distribution level? I assume the packager/maintainer for Libgcrypt on a given distribution should know whether the getrandom syscall is available on said distribution, so he could install a /etc/gcrypt/random.conf file with the only-urandom option.

Werner wrote:

we already use the getrandom system call if it is available

I am currently considering improvement of finalizer of libgcrypt, so, this matters.
Looking code, it would be better not to allocate and free the constant,
but use compile time constant data in .text section; Something like: const unsigned char ctr_null[DBRG_CTR_NULL_LEN].

To clarify: We already use the getrandom system call if it is available. To map /dev/random to /dev/urandom you can create a file /etc/gcrypt/random.conf with this line: