Page MenuHome GnuPG

macOS getentropy
Closed, ResolvedPublic

Description

D522 is a patch for macOS.

longlong part was fixed already in master (to be 1.9.1).

getentropy part should be handled.

Note that we can't directly apply the patch.

See: https://www.gnu.org/software/gnulib/manual/html_node/sys_002frandom_002eh.html
https://bugs.python.org/issue29057

Event Timeline

I realized that it's a bit difficult for macOS to use getentropy.

On some versions, even though it is available in the sys/random.h, it is actually not implemented (just a weak pointer).
See:

https://github.com/bitcoin/bitcoin/pull/10301
https://github.com/bitcoin/bitcoin/pull/15100
https://github.com/bitcoin/bitcoin/pull/15103

Just for the information, this library of Rust checks if the symbol definition is available (not NULL), and use getentropy in that case.
fall back to /dev/u?random

https://github.com/rust-random/getrandom/blob/master/src/macos.rs

gniibe added a project: libgcrypt.
gniibe moved this task from Backlog to For 1.9 on the libgcrypt board.

To support old macOS (< 10.12), I think that code should be something like this:

diff --git a/random/rndlinux.c b/random/rndlinux.c
index 04e2a464..f378a549 100644
--- a/random/rndlinux.c
+++ b/random/rndlinux.c
@@ -32,6 +32,10 @@
 #include <string.h>
 #include <unistd.h>
 #include <fcntl.h>
+#if defined(__APPLE__) && defined(__MACH__)
+extern int getentropy (void *buf, size_t buflen) __attribute__ ((weak_import));
+#define HAVE_GETENTROPY
+#endif
 #if defined(__linux__) || !defined(HAVE_GETENTROPY)
 #ifdef HAVE_SYSCALL
 # include <sys/syscall.h>
@@ -260,6 +264,9 @@ _gcry_rndlinux_gather_random (void (*add)(const void*, size_t,
        * not been properly seeded.  And it differs from /dev/random by never
        * blocking once the kernel is seeded.  */
 #if defined(HAVE_GETENTROPY) || defined(__NR_getrandom)
+#if defined(__APPLE__) && defined(__MACH__)
+      if (&getentropy != NULL)
+#endif
         {
           long ret;
           size_t nbytes;

We should not use system defined getentropy (on a build system >= 10.12), because it is defined with no 'weak_import' attribute.

I don't know when the symbol of getentropy was available on macOS.
For the old systems with no getentropy symbol, it won't work at all.

To support old macOS (< 10.12), I think that code should be something like this:

diff --git a/random/rndlinux.c b/random/rndlinux.c
index 04e2a464..f378a549 100644
We should not use system defined getentropy (on a build system >= 10.12), because it is defined with no 'weak_import' attribute.

Right now I cannot compile libgcrypt. According to theory I'd say your patch is not correct. GCC 4.2, which is used on Tiger and Leopard (Mac OS X 10.4.11 resp. 10.5.8), has these defines:

#define __MACH__ 1
#define __APPLE__ 1

and so does GCC 7.5, the alternative, when the source follows standards from this century. The version of Mac OS X or macOS can be checked by

#define __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ 10125

The number used expresses Mac OS X 10.12.5. Another option might be to use

if MAC_OS_X_VERSION_MIN_REQUIRED < 101200

The real problem with Tiger is that it misses a few C header files, and of them is required by recent versions of libgcrypt:

/opt/local/bin/gcc-apple-4.2 -DHAVE_CONFIG_H -I. -I..  -I../src -I../src -I/opt/local/include 
-I/opt/local/include -pipe -Os -std=gnu89 -arch ppc -fno-delete-null-pointer-checks -Wall -MT random.o -MD -MP -MF .deps/random.Tpo -c -o random.o random.c
random.c:509:19: error: spawn.h: No such file or directory
random.c: In function 'run_all_rng_tests':
random.c:551: warning: implicit declaration of function 'posix_spawn'
make[2]: *** [random.o] Error 1

sys/spawn.h has been provided as legacy support by MacPorts team, but it seems to be inadequate.

I don't know when the symbol of getentropy was available on macOS.

I think it was macOS Sierra, Version 10.12.x.

@ballapete Thank you for testing.

For the failure of compiling random.c, a patch has just been pushed to master:
rCfc901e978a0c: build: Check spawn.h for MacOS X Tiger.

I wrote:

I don't know when the symbol of getentropy was available on macOS.

You answered:

I think it was macOS Sierra, Version 10.12.x.

I know that the function implementation is available in 10.12 or later.
The problem here is that, in older version of 10.x (x < 12), the symbol itself is available, but there are no implementation actually.
If possible, I'd like to know when the symbol was available.

gniibe changed the task status from Open to Testing.Jan 27 2021, 5:04 AM

Push the change.
For older versions of MacOS X, I'll handle it later.

To support old macOS (< 10.12), I think that code should be something like this:

After libgcrypt 1.9.0 first built on Tiger and 'make check' succeeded I ran 'make clean' and applied the changes to random/rndlinux.c. A simple make succeeded without problem. 'make check' then ran into problems, spitting out many times messages messages like these:

make  check-TESTS
dyld: Symbol not found: _getentropy
  Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.0/src/.libs/libgcrypt.20.dylib
  Expected in: dynamic lookup

FAIL: version
dyld: Symbol not found: _getentropy
  Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.0/src/.libs/libgcrypt.20.dylib
  Expected in: dynamic lookup

It finished with:

FAIL: bench-slope
SKIP: hashtest-256g
=======================================
29 of 29 tests failed
(1 test was not run)
Please report to https://bugs.gnupg.org
=======================================

Thank you for your testing.

Suppose that getentropy is available in MacOS X 10.5 (which conforms to Single Unix Specification). If so, something like following should work:

diff --git a/random/rndlinux.c b/random/rndlinux.c
index a22db177..256f7403 100644
--- a/random/rndlinux.c
+++ b/random/rndlinux.c
@@ -33,9 +33,12 @@
 #include <unistd.h>
 #include <fcntl.h>
 #if defined(__APPLE__) && defined(__MACH__)
+#include <Availability.h>
+#ifdef __MAC_10_5
 extern int getentropy (void *buf, size_t buflen) __attribute__ ((weak_import));
 #define HAVE_GETENTROPY
 #endif
+#endif
 #if defined(__linux__) || !defined(HAVE_GETENTROPY)
 #ifdef HAVE_SYSCALL
 # include <sys/syscall.h>

It only enables use of getentropy for MacOS 10.5 or later.

In the next few days I'll be able to boot into Leopard, Mac OS X 10.5.8, to test this. Right now it seems that I need to fix a few problems with updated software sources…

I applied the two patches on Mac OS X 10.5.8, Leopard, to random/rndlinux.c, resulting in this unified diff:

  • random/rndlinux.c~ 2021-01-28 23:14:39.000000000 +0100

+++ random/rndlinux.c 2021-01-29 10:56:26.000000000 +0100
@@ -32,6 +32,13 @@
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
+#if defined(APPLE) && defined(MACH)
+#include <Availability.h>
+#ifdef MAC_10_5
+extern int getentropy (void *buf, size_t buflen)
attribute__ ((weak_import));
+#define HAVE_GETENTROPY
+#endif
+#endif
#if defined(linux) || !defined(HAVE_GETENTROPY)
#ifdef HAVE_SYSCALL

  1. include <sys/syscall.h>

@@ -260,6 +267,9 @@

  • not been properly seeded. And it differs from /dev/random by never
  • blocking once the kernel is seeded. */ #if defined(HAVE_GETENTROPY) || defined(__NR_getrandom)

+#if defined(APPLE) && defined(MACH)
+ if (&getentropy != NULL)
+#endif

{
  long ret;
  size_t nbytes;

At this state random/rndlinux.c compiled fine. I am letting run 'make check' – again it fails à la:

make check-TESTS
make[2]: Verzeichnis „/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.0/tests“ wird betreten
dyld: Symbol not found: _getentropy

Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.0/src/.libs/libgcrypt.20.dylib
Expected in: dynamic lookup

/bin/sh: line 5: 78501 Trace/BPT trap (core dumped) GCRYPT_IN_REGRESSION_TEST=1 ${dir}$tst
FAIL: version
dyld: Symbol not found: _getentropy

Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.0/src/.libs/libgcrypt.20.dylib
Expected in: dynamic lookup

/bin/sh: line 5: 78520 Trace/BPT trap (core dumped) GCRYPT_IN_REGRESSION_TEST=1 ${dir}$tst

Wouldn't it be better to move these failures as a single one into the configure script that it definitely can tell "This Mac has getentropy()"?

Wouldn't it be better to move these failures as a single one into the configure script that it definitely can tell "This Mac has getentropy()"?

For build from source, this works. Well, that will be the last option, if it will be found too difficult.

I have been trying to support the situation where a binary built by some versions of macOS can be used on different version (like Python and bitcoin, I referred).
To do that, I'd like to know, when the symbol getentropy was added.

To do that, I'd like to know, when the symbol getentropy was added.

So it would be sufficient to find all dylibs and check with nm whether they contain the symbol? On PPC Tiger and Leopard (Mac OS X 10.4.11/10.5.8) this can be performed by me. For later Mac OS X versions I would need to install these versions in a VM… (which might not work because the install programme is possibly checking whether it is guide by a hypervisor)

There is some (partly) good news: The function getentropy() is available in the packet manager MacPorts. It has a legacy support:

/opt/local/lib/libMacportsLegacySupport.dylib:single module: 00002a88 T __getentropy

This legacy support serves as a fall-back for modern functions not found in old systems. I don't whether guards exist for MacPorts…

I got hit of search by "$ld$weak$os10.11$_getentropy".
So, I guess that it's 10.11 which has _getentropy as weak symbol, and 10.12 or later has implementation.

So, the change against libgcrypt 1.9.1 will be:

diff --git a/random/rndlinux.c b/random/rndlinux.c
index a22db177..a7a78906 100644
--- a/random/rndlinux.c
+++ b/random/rndlinux.c
@@ -33,9 +33,12 @@
 #include <unistd.h>
 #include <fcntl.h>
 #if defined(__APPLE__) && defined(__MACH__)
+#include <Availability.h>
+#ifdef __MAC_10_11
 extern int getentropy (void *buf, size_t buflen) __attribute__ ((weak_import));
 #define HAVE_GETENTROPY
 #endif
+#endif
 #if defined(__linux__) || !defined(HAVE_GETENTROPY)
 #ifdef HAVE_SYSCALL
 # include <sys/syscall.h>

This approach is too simplistic. See Ryan Schmidt's and Joshua Root's comments in https://trac.macports.org/ticket/62278

Alright! Here is the error report from GCC:

/bin/sh ../libtool  --tag=CC   --mode=compile /opt/local/bin/gcc-apple-4.2 -DHAVE_CONFIG_H -I. -I..  -I../src -I../src -isystem/opt/local/include/LegacySupport -I/opt/local/include -I/opt/local/include -pipe -Os -std=gnu89 -arch ppc -fno-delete-null-pointer-checks -Wall -MT rndlinux.lo -MD -MP -MF .deps/rndlinux.Tpo -c -o rndlinux.lo rndlinux.c
libtool: compile:  /opt/local/bin/gcc-apple-4.2 -DHAVE_CONFIG_H -I. -I.. -I../src -I../src -isystem/opt/local/include/LegacySupport -I/opt/local/include -I/opt/local/include -pipe -Os -std=gnu89 -arch ppc -fno-delete-null-pointer-checks -Wall -MT rndlinux.lo -MD -MP -MF .deps/rndlinux.Tpo -c rndlinux.c  -fno-common -DPIC -o .libs/rndlinux.o
rndlinux.c:36:26: error: Availability.h: No such file or directory
make[2]: *** [rndlinux.lo] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.2/random'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.2'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.2'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_libgcrypt/libgcrypt/work/libgcrypt-1.9.2" && /usr/bin/make -w all 
Exit code: 2

I thought the simple question whether rndlinux.c is needed on Mac OS X would not need this background…

Tiger's /usr/include/AvailabilityMacros.h seems to provide what rndlinux.c was looking for: libgcrypt 1.9.2 just built!

So, I actually just filed an issue about this work: T5375, and then found this opposing task while following through on the various commits ;P... Apple actually forbids usage of getentropy in applications they publish to their App Store (citing ITMS-90338: Non-public API usage), and so there needs to be a way to disable this weak_import. FWIW, I'm not sure if this is only on iOS or on macOS as well (I haven't gotten around to trying to publish a macOS build with the new libgcrypt yet).

gniibe set External Link to https://trac.macports.org/ticket/62431.Aug 25 2021, 3:49 AM

Closing, as downstream ticket has been closed.