Page MenuHome GnuPG

Apple M1 and Symbol not found: __gcry_mpih_mul_1
Closed, ResolvedPublic

Description

Hi Everyone,

I'm testing GnuPG 2.2.27 on an Apple M1. That's the new Apple desktop with the ARMv8.2-a processor. It has Apple Clang 12.0.0 (Apple Clang versions do not follow LLVM Clang versioning). I'm using the latest config.guess and config.sub, and the triplet aarch64-apple-darwin. System Integrity Protection (SIP) is enabled.

make check is failing a number of tests:

/usr/bin/clang  -Wall -Wno-pointer-sign -Wpointer-arith -g2 -O2 -fno-common -arch arm64 -fPIC -pthread  -L/usr/local/lib -Wl,-rpath,@loader_path/../lib -Wl,-rpath,/usr/local/lib -L/usr/local/lib -L/usr/local/lib -L/usr/local/lib -o asschk asschk.o
gpgsm: WARNING: running with faked system time: 2002-12-02 13:29:59
gpgsm: keybox '/Users/jwalton/Build-Scripts/gnupg-2.2.27/tests/pubring.kbx' created
dyld: lazy symbol binding failed: Symbol not found: __gcry_mpih_mul_1
  Referenced from: /usr/local/lib/libgcrypt.20.dylib
  Expected in: flat namespace

On the Mac-mini with the M1:

% nm /usr/local/lib/libgcrypt.20.dylib | grep gcry_mpih_mul
0000000000075ed4 T __gcry_mpih_mul
                 U __gcry_mpih_mul_1
0000000000075af8 T __gcry_mpih_mul_karatsuba_case
00000000000753b4 T __gcry_mpih_mul_n
000000000007be64 T _gcry_mpih_mul_1

And on an Intel Mac-mini where GnuPG tests ok:

$ nm /usr/local/lib/libgcrypt.20.dylib | grep gcry_mpih_mul
00000000000bae90 T __gcry_mpih_mul
00000000000c1e4f T __gcry_mpih_mul_1
00000000000ba8f0 T __gcry_mpih_mul_karatsuba_case
00000000000b9fd0 T __gcry_mpih_mul_n

I'm not sure where to look for the difference. It may be in the way libgcrypt is being built. Or it may be in the way GnuPG is creating mangled names.

I also noticed other packages that depend on libgcrypt do not have a problem. The other packages include Emacs, libxslt, libmicrohttpd and ntbTLS.

Details

Version
2.2.27

Event Timeline

This is kind of a hack, but this patch:

--- mpi/aarch64/mpih-mul1.S
+++ mpi/aarch64/mpih-mul1.S
@@ -36,7 +36,11 @@
 
 .globl _gcry_mpih_mul_1
 ELF(.type  _gcry_mpih_mul_1,%function)
+.globl __gcry_mpih_mul_1
+ELF(.type  __gcry_mpih_mul_1,%function)
+
 _gcry_mpih_mul_1:
+__gcry_mpih_mul_1:
 	CFI_STARTPROC()
 	and	w5, w2, #3;
 	mov	x4, xzr;

Creates a second symbol with the double underscore. There does not seem to be code duplication:

% nm /usr/local/lib/libgcrypt.20.dylib | grep gcry_mpih_mul
0000000000075ed4 T __gcry_mpih_mul
000000000007be64 T __gcry_mpih_mul_1
0000000000075af8 T __gcry_mpih_mul_karatsuba_case
00000000000753b4 T __gcry_mpih_mul_n
000000000007be64 T _gcry_mpih_mul_1

The reason I chose a second duplicate symbol is, there's code in the field using _gcry_mpih_mul_1 because of existing code. The code should include __gcry_mpih_mul_1 for regular users, and _gcry_mpih_mul_1 for users who worked around the problem.

After giving fair warning, you should be able to remove _gcry_mpih_mul_1.

Here's the patch I am using for the Apple M1: libgcrypt-darwin.patch. The patch is public domain so anyone is free to use it.

The patch tested OK, and both libgcrypt and GnuPG both pass their self tests. The patch clears the missing symbols. The patch also clears this issue from the linker:

libtool: link: /usr/bin/clang -dynamiclib -Wl,-flat_namespace -Wl,-undefined -Wl,suppress -o .libs/libgcrypt.20.dylib  .libs/libgcrypt_la-visibility.o .libs/libgcrypt_la-misc.o .libs/libgcrypt_la-global.o .libs/libgcrypt_la-sexp.o .libs/libgcrypt_la-hwfeatures.o .libs/libgcrypt_la-stdmem.o .libs/libgcrypt_la-secmem.o .libs/libgcrypt_la-missing-string.o .libs/libgcrypt_la-fips.o .libs/libgcrypt_la-hmac256.o .libs/libgcrypt_la-context.o .libs/libgcrypt_la-hwf-arm.o   -Wl,-force_load,../cipher/.libs/libcipher.a -Wl,-force_load,../random/.libs/librandom.a -Wl,-force_load,../mpi/.libs/libmpi.a -Wl,-force_load,../compat/.libs/libcompat.a  -L/usr/local/lib /usr/local/lib/libgpg-error.dylib -ldl -lpthread  -O2 -arch arm64 -pthread -Wl,-rpath -Wl,@loader_path/../lib -Wl,-rpath -Wl,/usr/local/lib   -pthread -install_name  /usr/local/lib/libgcrypt.20.dylib -compatibility_version 24 -current_version 24.2 -Wl,-single_module
ld: warning: arm64 function not 4-byte aligned: _gcry_mpih_add_n from ../mpi/.libs/libmpi.a(mpih-add1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Loop from ../mpi/.libs/libmpi.a(mpih-add1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Large_loop from ../mpi/.libs/libmpi.a(mpih-add1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Lend from ../mpi/.libs/libmpi.a(mpih-add1-asm.o)
ld: warning: arm64 function not 4-byte aligned: _gcry_mpih_sub_n from ../mpi/.libs/libmpi.a(mpih-sub1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Loop from ../mpi/.libs/libmpi.a(mpih-sub1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Large_loop from ../mpi/.libs/libmpi.a(mpih-sub1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Lend from ../mpi/.libs/libmpi.a(mpih-sub1-asm.o)
ld: warning: arm64 function not 4-byte aligned: _gcry_mpih_mul_1 from ../mpi/.libs/libmpi.a(mpih-mul1-asm.o)
ld: warning: arm64 function not 4-byte aligned: __gcry_mpih_mul_1 from ../mpi/.libs/libmpi.a(mpih-mul1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Loop from ../mpi/.libs/libmpi.a(mpih-mul1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Large_loop from ../mpi/.libs/libmpi.a(mpih-mul1-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Lend from ../mpi/.libs/libmpi.a(mpih-mul1-asm.o)
ld: warning: arm64 function not 4-byte aligned: _gcry_mpih_addmul_1 from ../mpi/.libs/libmpi.a(mpih-mul2-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Loop from ../mpi/.libs/libmpi.a(mpih-mul2-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Large_loop from ../mpi/.libs/libmpi.a(mpih-mul2-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Lend from ../mpi/.libs/libmpi.a(mpih-mul2-asm.o)
ld: warning: arm64 function not 4-byte aligned: _gcry_mpih_submul_1 from ../mpi/.libs/libmpi.a(mpih-mul3-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Loop from ../mpi/.libs/libmpi.a(mpih-mul3-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Large_loop from ../mpi/.libs/libmpi.a(mpih-mul3-asm.o)
ld: warning: arm64 function not 4-byte aligned: .Loop_end from ../mpi/.libs/libmpi.a(mpih-mul3-asm.o)
JW updated the task description. (Show Details)

This patch should work if configure properly detects need for extra underscore on C symbols:

Generated file mpi/sysdep.h defines C_SYMBOL_NAME macro which should have the underscore handled.

Thanks @jukivili.

This patch should work if configure properly detects need for extra underscore on C symbols:

-.globl _gcry_mpih_mul_1
-ELF(.type  _gcry_mpih_mul_1,%function)
-_gcry_mpih_mul_1:
+.globl C_SYMBOL_NAME(_gcry_mpih_mul_1)
+ELF(.type  C_SYMBOL_NAME(_gcry_mpih_mul_1),%function)
+C_SYMBOL_NAME(_gcry_mpih_mul_1):

Do you want to remove the single underscore names? That may require a major version bump because a symbol is disappearing.

These functions are internal to library and, for example, on linux/windows builds are not externally available.

Maybe "dyld: lazy symbol binding failed: Symbol not found: gcry_mpih_mul_1" is generated when upper level MPI functions in libgcrypt attempt to call gcry_mpih_mul_1.

Do what ever you want with _gcry prefixed functions - this is never considered an API or ABI break. There are some exceptions for internal functions used by macros but those are clearly marked.

werner triaged this task as Normal priority.Apr 1 2021, 11:02 AM
werner added a project: MacOS.
gniibe claimed this task.