Install from source fails on OSX

[schlegelp]

• pyn5 version: 1.1.1
• Python version: 3.7.5
• Operating System: OSX 10.14.6 (Mojave)

Description

pip3 install py5n fails with a nondescript error. I have cargo 1.56.0, ructc 1.56.1 and maturin 0.12.3. Quite likely an issue on my end.

What I Did

Philipps-MacBook-Pro-2:~ philipps$ pip3 install pyn5 -U
Collecting pyn5
  Using cached pyn5-1.1.1.tar.gz (23 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting h5py_like>=0.6.0
  Using cached h5py_like-0.6.0-py3-none-any.whl (22 kB)
Requirement already satisfied, skipping upgrade: numpy in ./.pyenv/versions/3.7.5/lib/python3.7/site-packages (from pyn5) (1.21.4)
Building wheels for collected packages: pyn5
  Building wheel for pyn5 (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/philipps/.pyenv/versions/3.7.5/bin/python3.7 /Users/philipps/.pyenv/versions/3.7.5/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/b1/1fbq04gx1vg344_ctkmv52d00000gn/T/tmpau5u3800
       cwd: /private/var/folders/b1/1fbq04gx1vg344_ctkmv52d00000gn/T/pip-install-adl_g2fa/pyn5
  Complete output (66 lines):
     Compiling proc-macro2 v1.0.32
     Compiling unicode-xid v0.2.2
     Compiling syn v1.0.82
     Compiling autocfg v1.0.1
     Compiling libc v0.2.108
     Compiling serde_derive v1.0.130
     Compiling serde v1.0.130
     Compiling proc-macro-hack v0.5.19
     Compiling cfg-if v1.0.0
     Compiling rawpointer v0.2.1
     Compiling parking_lot_core v0.8.5
     Compiling cc v1.0.72
     Compiling pkg-config v0.3.22
     Compiling unindent v0.1.7
     Compiling ryu v1.0.6
     Compiling crc32fast v1.2.2
     Compiling inventory v0.1.11
     Compiling scopeguard v1.1.0
     Compiling ndarray v0.13.1
     Compiling pyo3 v0.11.1
     Compiling serde_json v1.0.72
     Compiling adler v1.0.2
     Compiling either v1.6.1
     Compiling itoa v0.4.8
     Compiling semver-parser v0.7.0
     Compiling same-file v1.0.6
     Compiling cfg-if v0.1.10
     Compiling byteorder v1.4.3
     Compiling instant v0.1.12
     Compiling matrixmultiply v0.3.2
     Compiling matrixmultiply v0.2.4
     Compiling lock_api v0.4.5
     Compiling num-traits v0.2.14
     Compiling num-integer v0.1.44
     Compiling num-complex v0.2.4
     Compiling miniz_oxide v0.4.4
     Compiling itertools v0.8.2
     Compiling walkdir v2.3.2
     Compiling semver v0.9.0
     Compiling fs2 v0.4.3
     Compiling quote v1.0.10
     Compiling bzip2-sys v0.1.11+1.0.8
     Compiling paste-impl v0.1.18
     Compiling flate2 v1.0.22
     Compiling paste v0.1.18
     Compiling num-complex v0.4.0
     Compiling ndarray v0.15.4
     Compiling bzip2 v0.4.3
     Compiling pyo3-derive-backend v0.11.1
     Compiling ghost v0.1.2
     Compiling inventory-impl v0.1.11
     Compiling indoc-impl v0.3.6
     Compiling ctor v0.1.21
     Compiling indoc v0.3.6
     Compiling pyo3cls v0.11.1
     Compiling smallvec v1.7.0
     Compiling parking_lot v0.11.2
     Compiling n5 v0.7.6
     Compiling numpy v0.11.0
     Compiling pyn5 v1.1.1 (/private/var/folders/b1/1fbq04gx1vg344_ctkmv52d00000gn/T/pip-install-adl_g2fa/pyn5)
  error: could not compile `pyn5` due to 21 previous errors; 1 warning emitted
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit code: 101": `cargo rustc --message-format json --manifest-path Cargo.toml --lib --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup`
  Running `maturin pep517 build-wheel -i python`
  Error: Command '['maturin', 'pep517', 'build-wheel', '-i', 'python']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for pyn5
Failed to build pyn5
ERROR: Could not build wheels for pyn5 which use PEP 517 and cannot be installed directly

[pattonw]

hmm, I'd guess its a problem with using OSX. I don't see anything else in the log that indicates what might be going wrong.
I've only written tests and put wheels on pypi for linux. There's an issue for creating wheels for other operating systems: #147
But I haven't looked into it much yet.

[schlegelp]

Strangely cloning the repo and running maturin pep517 build-wheel -I python manually works just fine.

[william-silversmith]

I'm having a build problem on MacOS with Apple Silicon where arm64 wheels are being build under a rosetta2 terminal instead of x86_64. It would be ideal if maturin would default to building universal2 wheels.

@schlegelp I inspected the source dist on pypi and found that there is a difference in the directory structure where in the cloned repo we have python/pyn5/... and in the source dist it's just pyn5/.... That's probably the issue?

[william-silversmith]

Looks like these targets can be generated with:

rustup target add x86_64-apple-darwin

And in the make dist command:

dist: clean ## builds source and wheel package
	maturin build -i python3.8 -i python3.9 -i python3.10 --release --universal2 && \
	ls -l $(DIST_DIR)

This will generate:

(pyn5) $ ls target/wheels
pyn5-1.1.1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
pyn5-1.1.1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
pyn5-1.1.1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl

[william-silversmith]

Here are some pre-compiled universal2 binaries for Apple Silicon (and Intel) users.

Archive.zip