Apple Silicon Binaries

[william-silversmith]

Hi all,

Thanks for this great library! I found that it is a little difficult (but not too difficult) to install on Apple Silicon as it requires installing rust nightly to get maturin installed.

Would it be possible to release binaries for this platform?

Thanks!

[ijl]

It's dependent on the availability of CI. I would not expect macOS aarch64 runners to be available soon. I see some effort on cross compilation in maturin. In that case CI would not be able to test the artifact. I think the risk of an issue unique to macOS aarch64 is low and users unusually dependent on binary wheels given the Rust toolchain, so that's a fine trade off. If someone would like to help I suggest looking at setting up cross compilation in CI.

[muuvmuuv]

Hey, is this related to orjson or more to maturin? I am running an Mac mini M1.

Collecting orjson==3.4.8
  Downloading orjson-3.4.8.tar.gz (658 kB)
     |████████████████████████████████| 658 kB 4.4 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/marvin/.pyenv/versions/3.9.1/envs/m8-svc-device/bin/python3.9 /Users/marvin/.pyenv/versions/3.9.1/envs/m8-svc-device/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/p8/v45460fx40sdsqvz89vhmwr80000gn/T/tmp9ocl0ygz
         cwd: /private/var/folders/p8/v45460fx40sdsqvz89vhmwr80000gn/T/pip-install-id1bqiq5/orjson
    Complete output (11 lines):
    💥 maturin failed
      Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
      Caused by: `cargo metadata` exited with an error: error: failed to run `rustc` to learn about target-specific information

    Caused by:
      process didn't exit successfully: `rustc - --crate-name ___ --print=file-names -Z mutable-noalias --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 1)
      --- stderr
      error: the option `Z` is only accepted on the nightly compiler
    Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/private/var/folders/p8/v45460fx40sdsqvz89vhmwr80000gn/T/pip-modern-metadata-1ns76lu_', '--interpreter', '/Users/marvin/.pyenv/versions/3.9.1/envs/m8-svc-device/bin/python3.9', '--manylinux=off', '--strip=on']' returned non-zero exit status 1.
    Checking for Rust toolchain....
    Running `maturin pep517 write-dist-info --metadata-directory /private/var/folders/p8/v45460fx40sdsqvz89vhmwr80000gn/T/pip-modern-metadata-1ns76lu_ --interpreter /Users/marvin/.pyenv/versions/3.9.1/envs/m8-svc-device/bin/python3.9 --manylinux=off --strip=on`
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/marvin/.pyenv/versions/3.9.1/envs/m8-svc-device/bin/python3.9 /Users/marvin/.pyenv/versions/3.9.1/envs/m8-svc-device/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/p8/v45460fx40sdsqvz89vhmwr80000gn/T/tmp9ocl0ygz Check the logs for full command output.

[william-silversmith]

You need to run rustup default nightly and rustup update.

[messense]

Opened #164 to build macOS universal2 wheel which supports both Intel and Apple Silicon Macs.

[ijl]

https://github.com/ijl/orjson/releases/tag/3.5.1

[messense]

Something is wrong with the deploy step in CI, universal2 wheels didn't get uploaded to pypi.

[messense]

twine upload --non-interactive --skip-existing "$1"

I think $1 is the cause, maybe it should be $@

[ijl]

Ok, they've been uploaded now.

[messense]

Note that you need to use pip version 21.0.1 and later to install universal2 wheels.

❯ pip install orjson
Collecting orjson
  Downloading orjson-3.5.1-cp38-cp38-macosx_10_9_universal2.whl (435 kB)
     |████████████████████████████████| 435 kB 856 kB/s
Installing collected packages: orjson
Successfully installed orjson-3.5.1

[sd2k]

It looks like some of these builds are timing out while compiling the Rust dependencies (possibly the custom serde_json dep?): https://cloud.drone.io/ijl/orjson/429/1/2. This is causing aarch64 wheel availability to be a bit spotty for some versions. 🤔

[ijl]

Yes, the timeouts are intermittent since a week ago with no compiler change. It seems a Drone CI issue. I don't know of an alternative aarch64 CI provider.

[messense]

If you can accept cross compiling to aarch64, maturin-action can do it on GitHub Actions, then we can run tests in QEMU with https://github.com/uraimo/run-on-arch-action