Pip install om M1 Mac works after some pain

Installation
1

Hello,

On an M1 Mac mini running MacOS 11.2.2, having installed one of the Apple Silicon builds, Python would report that I was “Trying to load an unsigned library”. Turns out that it was not so hard to solve. At least now it appears to work after I executed these commands

ls -1 *.so | awk '{print "codesign -s -", $1}' | sh
ls -1 *.dylib | awk '{print "codesign -s -", $1}' | sh

from

/Users//Library/Python/3.9/lib/python/site-packages

Now I just need to figure out how to use Panda3d :slight_smile:

2

Hi, welcome to the community!

Thanks for this. Interesting, what does codesign -s - do? -s means to sign, but using what certificate?

3

Thanks.

As I understand ‘-s -’ signs with the ad hoc signature. I looked at some other Python libraries in the site-packages directory and they had been signed ‘adhoc’ whereas all panda3d libraries were unsigned. I suppose it comes down to specifics of the build process, but no actual certificate was provided by me.

4

Interesting. Could you give me an example of another pip package you installed that is codesigned? I wonder about their build process.

For the record, this is tracked as issue here now:

5

I first looked at numpy which was ‘adhoc’ signed. Also, I develop pygel3d, and you can see below that it is also ‘adhoc’ code signed. I did nothing to achieve this. It is a very standard build process using CMake. However, the compiler is ‘clang’. If Panda3d is compiled and linked with a different tool chain that could be the reason.

% codesign -d -v libPyGEL.dylib
Executable=/Users/andreas/Library/Python/3.9/lib/python/site-packages/pygel3d/libPyGEL.dylib
Identifier=libPyGEL.dylib
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=48263 flags=0x20002(adhoc,linker-signed) hashes=1505+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
6

I just cloned and built that GEL repository and it doesn’t appear to be signed:

$ codesign -d -v libPyGEL.dylib
libPyGEL.dylib: code object is not signed at all

Same when I build the wheel and unzip it:

$ unzip PyGEL3D-0.1.4-py3-none-any.whl
$ codesign -d -v pygel3d/libPyGEL.dylib
pygel3d/libPyGEL.dylib: code object is not signed at all

What might I be missing?

7

Some more experimentation shows that invoking with the linker with -arch arm64 does generate the signature, but adding -arch arm64 -arch x86_64 (which is what we do to create a universal wheel) doesn’t. How odd.

8

Ok. I was mistaken. I can confirm that the dylib in the GEL wheel on PyPI does not have a signature - I get the same result as you. The version I tested was compiled on my local machine (M1 Mac mini). The PyPI version is compiled on GitHub for X86_64 using GitHub Actions. Honestly, I thought that there would be no difference regarding signature, but turns out I was wrong. It is the same version of Xcode but GitHub does not run MacOS 11 yet.

So, the question is why on earth there is a signature if it is compiled for arm64 but not if it is compiled x86_64 or for both architectures. For now, I guess you could run codesign afterwards, but it seems to be a bug in Apple’s toolchain.

Does the library load on x86_64 if it is signed?

9

I figured out that it’s because the fat binary as a whole isn’t codesigned, only the embedded arm64 binary, as can be confirmed using lipo -extract arm64.

This appeared to be a red herring, the problem is that we apparently have a codesign --strip-signature in our process of building a wheel:

We ought to replace this with a codesign -s - step when building for arm64.

10

I think this ought to fix it:

Could you please try this build?
https://buildbot.panda3d.org/downloads/576c25b3eab8d3a3501379801fac8c934efe624a/panda3d-1.10.9.dev130-cp39-cp39-macosx_11_0_universal2.whl

11

It works. That was fast :slight_smile:

12

Great! Thanks for testing, and thanks for helping me figure this out.

13

I fear the whl signatures are in a bad state — this may correspond to when I had to downgrade from doing osx_11_0 builds to osx_10_14 builds.

% python3 setup_glb.py build_apps
running build_apps
Building platforms: macosx_11_0_x86_64
Gathering wheels for platform: macosx_11_0_x86_64
Looking in indexes: https://pypi.org/simple, https://archive.panda3d.org/simple/opt, https://archive.panda3d.org/thirdparty
Collecting panda3d
  Using cached https://buildbot.panda3d.org/downloads/v1.10.9/opt/panda3d-1.10.9%2Bopt-cp39-cp39-macosx_11_0_universal2.whl (111.0 MB)
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    panda3d from https://buildbot.panda3d.org/downloads/v1.10.9/opt/panda3d-1.10.9%2Bopt-cp39-cp39-macosx_11_0_universal2.whl#md5=6719fa4ca2fc24591b4a7638d2eebc87 (from -r <redacted>/requirements.txt (line 1)):
        Expected md5 6719fa4ca2fc24591b4a7638d2eebc87
             Got        651b1c61dd502d24a26254f955af115a
14

I think this should be fixed, let me know if it happens again.