make check fails for libgcrypt on Apple Silicon / ARM Mac
Testing, NormalPublic

Description

On a Macbook Pro with the new Apple Silicon M1 chip, compiling libgcrypt finishes successfully, however, make check fails the random test (all other tests PASS) with the following error:

dyld: Library not loaded: /usr/local/lib/libgcrypt.20.dylib
  Referenced from: /Users/mbirth/Development/libgcrypt/libgcrypt-1.8.7/tests/.libs/random
  Reason: image not found
random: running '/Users/mbirth/Development/libgcrypt/libgcrypt-1.8.7/tests/.libs/random --in-recursion --early-rng-check' failed
FAIL: random

This also fails the test when installing the Homebrew package, so the package is never actually installed. See https://github.com/Homebrew/brew/issues/7857#issuecomment-735269479 .

mbirth created this task.Nov 28 2020, 11:43 PM
mbirth updated the task description. (Show Details)Nov 29 2020, 1:14 PM
gniibe added a subscriber: gniibe.Nov 30 2020, 2:51 AM
This comment was removed by gniibe.
gniibe added projects: MacOS, Info Needed.EditedNov 30 2020, 7:30 AM

IIUC, for the build of Homebrew, it is the issue of in: https://github.com/Homebrew/homebrew-core/commit/e7da1e2157b2e8373c3b39ea6398f51588ea537c

It changes the library path by the script, so that random can run correctly. I think that for some reason (of the upgrade), lib/"libgcrypt.20.dylib" perhaps would be changed to different place.

Please give us information for your own manual build of libgcrypt (not by the Homebrew).

After doing:

sudo ln -s /Users/mbirth/Development/libgcrypt/libgcrypt-1.8.7/src/.libs/libgcrypt.20.dylib /usr/local/lib/libgcrypt.20.dylib

All tests pass now.

Just to be clear: I didn't use any parameters when compiling. All I did after untarring was:

./configure
make

After disabling SIP, now all checks pass without having the library symlinked to /usr/local/lib. So it might be T2056: libgcrypt: make check fails "random" test on OS X 10.11 with link error after all.

gniibe added a comment.EditedDec 1 2020, 4:18 AM

BTW, I'm not sure if the claim in T5009#136688 is correct.

If it is the system interpreter invocation (/bin/sh) which sanitizes the environment variable, following patch would work:

diff --git a/tests/random.c b/tests/random.c
index 22927a61..a1244664 100644
--- a/tests/random.c
+++ b/tests/random.c
@@ -505,6 +505,7 @@ check_drbg_reinit (void)
 }
 
 
+#ifdef HAVE_W32_SYSTEM
 /* Because we want to check initialization behaviour, we need to
    fork/exec this program with several command line arguments.  We use
    system, so that these tests work also on Windows.  */
@@ -559,7 +560,64 @@ run_all_rng_tests (const char *program)
 
   free (cmdline);
 }
+#else
+#include <spawn.h>
+
+static void
+run_all_rng_tests (const char *program)
+{
+  static const char *options[][2] = {
+    { "--early-rng-check",     NULL },
+    { "--early-rng-check",     "--prefer-standard-rng" },
+    { "--early-rng-check",     "--prefer-fips-rng" },
+    { "--early-rng-check",     "--prefer-system-rng" },
+    { "--prefer-standard-rng", NULL },
+    { "--prefer-fips-rng",     NULL },
+    { "--prefer-system-rng",   NULL },
+    { NULL, NULL }
+  };
+  int idx;
+  char *argv[8];
 
+  for (idx=0; options[idx][0]; idx++)
+    {
+      int i;
+      pid_t pid;
+      int status;
+
+      if (verbose)
+        info ("now running with options '%s%s%s'\n",
+              options[idx][0],
+              options[idx][1] ? " " : "",
+              options[idx][1] ? options[idx][1] : "");
+
+      i = 0;
+      argv[i++] = xstrdup (program);
+      argv[i++] = xstrdup ("--in-recursion");
+      argv[i++] = xstrdup ("--verbose");
+      argv[i++] = xstrdup ("--debug");
+      argv[i++] = xstrdup ("--progress");
+      argv[i++] = xstrdup (options[idx][0]);
+      if (options[idx][1])
+        argv[i++] = xstrdup (options[idx][1]);
+      argv[i++] = NULL;
+
+      if (posix_spawn (&pid, program, NULL, NULL, argv, environ))
+        die ("spawning '%s' failed\n", program);
+
+      if (waitpid (pid, &status, 0) < 0)
+        die ("waitpid for '%s' failed\n", program);
+
+      if (WIFEXITED (status) && WEXITSTATUS (status))
+        die ("running '%s' failed with %d\n", program, WEXITSTATUS (status));
+      else if (!WIFEXITED (status))
+        die ("running '%s' failed\n", program);
+
+      while (i)
+        xfree (argv[--i]);
+    }
+}
+#endif
 
 static void
 run_benchmark (void)
werner added a subscriber: werner.Dec 1 2020, 9:08 AM

The problem is that posix_spawn is not portable enough for libgcrypt. It is really time that we move the spawn functions from gnupg to gpgrt so that we can use them also in Libgcrypt.

mbirth added a comment.Dec 1 2020, 7:05 PM

After applying @gniibe 's patch:

gcc -DHAVE_CONFIG_H -I. -I..  -I../src -I../src  -I/opt/homebrew/opt/libgpg-error/include -g -O2 -fno-delete-null-pointer-checks -Wall -MT random.o -MD -MP -MF .deps/random.Tpo -c -o random.o random.c
random.c:605:57: error: use of undeclared identifier 'environ'
      if (posix_spawn (&pid, program, NULL, NULL, argv, environ))
                                                        ^
1 error generated.
make[1]: *** [random.o] Error 1
werner added a comment.Dec 1 2020, 8:51 PM

Put

extern char **environ;

after the the include directives.

mbirth added a comment.Dec 1 2020, 9:44 PM

Put

extern char **environ;

after the the include directives.

That did it!

====================
All 27 tests passed
(1 test was not run)
====================

I created D513: Support macOS build with SIP by using posix_spawn in tests/random, which is more conservative; It only affects build under macOS.

I agree that no posix_spawn in libgcrypt but having spawning a process in libgpg-error is good.

Note that the patch doesn't use use of posix_spawn in the libgcrypt library proper, but only in the test program for macOS.
If it would be difficult for libgcrypt upstream, macOS can use D513 for their build.

So, I'm going to push D513 to both of 1.8 and master (to be 1.9).

gniibe changed the task status from Open to Testing.Dec 3 2020, 3:11 AM
gniibe triaged this task as Normal priority.