Page MenuHome GnuPG

make check fails for libgcrypt on Apple Silicon / ARM Mac
Closed, ResolvedPublic

Description

On a Macbook Pro with the new Apple Silicon M1 chip, compiling libgcrypt finishes successfully, however, make check fails the random test (all other tests PASS) with the following error:

dyld: Library not loaded: /usr/local/lib/libgcrypt.20.dylib
  Referenced from: /Users/mbirth/Development/libgcrypt/libgcrypt-1.8.7/tests/.libs/random
  Reason: image not found
random: running '/Users/mbirth/Development/libgcrypt/libgcrypt-1.8.7/tests/.libs/random --in-recursion --early-rng-check' failed
FAIL: random

This also fails the test when installing the Homebrew package, so the package is never actually installed. See https://github.com/Homebrew/brew/issues/7857#issuecomment-735269479 .

Event Timeline

This comment was removed by gniibe.

IIUC, for the build of Homebrew, it is the issue of in: https://github.com/Homebrew/homebrew-core/commit/e7da1e2157b2e8373c3b39ea6398f51588ea537c

It changes the library path by the script, so that random can run correctly. I think that for some reason (of the upgrade), lib/"libgcrypt.20.dylib" perhaps would be changed to different place.

Please give us information for your own manual build of libgcrypt (not by the Homebrew).

After doing:

sudo ln -s /Users/mbirth/Development/libgcrypt/libgcrypt-1.8.7/src/.libs/libgcrypt.20.dylib /usr/local/lib/libgcrypt.20.dylib

All tests pass now.

Just to be clear: I didn't use any parameters when compiling. All I did after untarring was:

./configure
make

After disabling SIP, now all checks pass without having the library symlinked to /usr/local/lib. So it might be T2056: libgcrypt: make check fails "random" test on OS X 10.11 with link error after all.

BTW, I'm not sure if the claim in T5009#136688 is correct.

If it is the system interpreter invocation (/bin/sh) which sanitizes the environment variable, following patch would work:

diff --git a/tests/random.c b/tests/random.c
index 22927a61..a1244664 100644
--- a/tests/random.c
+++ b/tests/random.c
@@ -505,6 +505,7 @@ check_drbg_reinit (void)
 }
 
 
+#ifdef HAVE_W32_SYSTEM
 /* Because we want to check initialization behaviour, we need to
    fork/exec this program with several command line arguments.  We use
    system, so that these tests work also on Windows.  */
@@ -559,7 +560,64 @@ run_all_rng_tests (const char *program)
 
   free (cmdline);
 }
+#else
+#include <spawn.h>
+
+static void
+run_all_rng_tests (const char *program)
+{
+  static const char *options[][2] = {
+    { "--early-rng-check",     NULL },
+    { "--early-rng-check",     "--prefer-standard-rng" },
+    { "--early-rng-check",     "--prefer-fips-rng" },
+    { "--early-rng-check",     "--prefer-system-rng" },
+    { "--prefer-standard-rng", NULL },
+    { "--prefer-fips-rng",     NULL },
+    { "--prefer-system-rng",   NULL },
+    { NULL, NULL }
+  };
+  int idx;
+  char *argv[8];
 
+  for (idx=0; options[idx][0]; idx++)
+    {
+      int i;
+      pid_t pid;
+      int status;
+
+      if (verbose)
+        info ("now running with options '%s%s%s'\n",
+              options[idx][0],
+              options[idx][1] ? " " : "",
+              options[idx][1] ? options[idx][1] : "");
+
+      i = 0;
+      argv[i++] = xstrdup (program);
+      argv[i++] = xstrdup ("--in-recursion");
+      argv[i++] = xstrdup ("--verbose");
+      argv[i++] = xstrdup ("--debug");
+      argv[i++] = xstrdup ("--progress");
+      argv[i++] = xstrdup (options[idx][0]);
+      if (options[idx][1])
+        argv[i++] = xstrdup (options[idx][1]);
+      argv[i++] = NULL;
+
+      if (posix_spawn (&pid, program, NULL, NULL, argv, environ))
+        die ("spawning '%s' failed\n", program);
+
+      if (waitpid (pid, &status, 0) < 0)
+        die ("waitpid for '%s' failed\n", program);
+
+      if (WIFEXITED (status) && WEXITSTATUS (status))
+        die ("running '%s' failed with %d\n", program, WEXITSTATUS (status));
+      else if (!WIFEXITED (status))
+        die ("running '%s' failed\n", program);
+
+      while (i)
+        xfree (argv[--i]);
+    }
+}
+#endif
 
 static void
 run_benchmark (void)

The problem is that posix_spawn is not portable enough for libgcrypt. It is really time that we move the spawn functions from gnupg to gpgrt so that we can use them also in Libgcrypt.

After applying @gniibe 's patch:

gcc -DHAVE_CONFIG_H -I. -I..  -I../src -I../src  -I/opt/homebrew/opt/libgpg-error/include -g -O2 -fno-delete-null-pointer-checks -Wall -MT random.o -MD -MP -MF .deps/random.Tpo -c -o random.o random.c
random.c:605:57: error: use of undeclared identifier 'environ'
      if (posix_spawn (&pid, program, NULL, NULL, argv, environ))
                                                        ^
1 error generated.
make[1]: *** [random.o] Error 1

Put

extern char **environ;

after the the include directives.

Put

extern char **environ;

after the the include directives.

That did it!

====================
All 27 tests passed
(1 test was not run)
====================

I created D513: Support macOS build with SIP by using posix_spawn in tests/random, which is more conservative; It only affects build under macOS.

I agree that no posix_spawn in libgcrypt but having spawning a process in libgpg-error is good.

Note that the patch doesn't use use of posix_spawn in the libgcrypt library proper, but only in the test program for macOS.
If it would be difficult for libgcrypt upstream, macOS can use D513 for their build.

So, I'm going to push D513 to both of 1.8 and master (to be 1.9).

gniibe changed the task status from Open to Testing.Dec 3 2020, 3:11 AM
gniibe triaged this task as Normal priority.
gniibe claimed this task.

Also support older MacOS X, which has no posix_spawn.

Sorry to dig up an old report...

This looks like System Integrity Protection (SIP) scrubbing certain environmental variables. The trick to the workaround is, have the Makefile set DYLD_LIBRARY_PATH before it calls the test runner.

This patch clears the issue so you don't have to disable SIP: libgcrypt.patch. Here's the interesting part:

--- tests/Makefile.in
+++ tests/Makefile.in
@@ -942,6 +942,10 @@
 check-TESTS: $(TESTS)
 	@failed=0; all=0; xfail=0; xpass=0; skip=0; \
 	srcdir=$(srcdir); export srcdir; \
+	gcrypt_libdir=`dirname $$PWD`/src/.libs; \
+	LD_LIBRARY_PATH=`echo "$$gcrypt_libdir:$$LD_LIBRARY_PATH" | $(SED) 's/:*$$//g'`; \
+	DYLD_LIBRARY_PATH=`echo "$$gcrypt_libdir:$$DYLD_LIBRARY_PATH" | $(SED) 's/:*$$//g'`; \
+	export LD_LIBRARY_PATH; export DYLD_LIBRARY_PATH; \
 	list=' $(TESTS) '; \
 	$(am__tty_colors); \
 	if test -n "$$list"; then \

The commands just prepend the proper directory to LD_LIBRARY_PATH and DYLD_LIBRARY_PATH. The sed removes any trailing : in case the variable was previously empty. Empty paths are removed for one of the BSDs (I think NetBSD is the one that acts funny with empty paths).

dirname is used because you are actually in the tests/ directory, and you want to trim that off. dirname returns a full path like /Users/jdoe/libgcrypt-1.9.2. Then you add src/.libs to it to get a path of /Users/jdoe/libgcrypt-1.9.2/src/.libs.

The Makefile trick does not always work when SIP is in effect because SIP will scrub the variables if you call a system program, like /bin/sh. You have to call your program, and not a system program.


Here's an example where the makefile trick does not work:

check:
    DYLD_LIBRARY_PATH=`echo "$$gcrypt_libdir:$$DYLD_LIBRARY_PATH" | $(SED) 's/:*$$//g'`; \
    export DYLD_LIBRARY_PATH; \
    /bin/sh gcrypt_test_runner.sh --all_tests

The trick would not work in this instance because SIP would clear DYLD_LIBRARY_PATH when /bin/sh is called. To fix it in this instance, gcrypt_test_runner.sh would have to set DYLD_LIBRARY_PATH.