Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: install gufo-snmp on Mac (m3) fails on rust issue #10

Open
nielsvanhooy opened this issue Feb 27, 2024 · 12 comments
Open

BUG: install gufo-snmp on Mac (m3) fails on rust issue #10

nielsvanhooy opened this issue Feb 27, 2024 · 12 comments
Labels
bug Something isn't working

Comments

@nielsvanhooy
Copy link

nielsvanhooy commented Feb 27, 2024

Describe the issue:

tried installing the package on my MacBook Pro m3 pro laptop but it ran into a problem.
I have no issues on another laptop that runs ubuntu

Reproduce the code example:

pdm add gufo-snmp

Error message:

(inetutils-python-3.12) sh-3.2# pdm add gufo-snmp
Adding packages to default dependencies: gufo-snmp
🔒 Lock successful
Changes are written to pyproject.toml.
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove

  ✖ Install gufo-snmp 0.5.0 failed
Retry failed jobs
  ✖ Install gufo-snmp 0.5.0 failed

ERRORS:
add gufo-snmp failed:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 285, in install_candidate
    self.manager.install(can)
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/manager.py", line 34, in install
    dist_info = installer(str(prepared.build()), self.environment, prepared.direct_url())
                              ^^^^^^^^^^^^^^^^
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/models/candidates.py", line 418, in build
    self.wheel = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/wheel.py", line 28, in build
    filename = self._hook.build_wheel(out_dir, config_settings, metadata_directory)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 209, in build_wheel
    return self._call_hook('build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 261, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 107, in log_subprocessor
    raise build_error(e) from None
pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
adding license file 'LICENSE.md'
writing manifest file 'src/gufo_snmp.egg-info/SOURCES.txt'
running build_ext
running build_rust
cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- -C link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/_fast.cpython-312-darwin.so
error: failed to parse manifest at `/private/tmp/pdm-build-bffi8i33/Cargo.toml`

Caused by:
  can't find `cri_decode` bench at `benches/cri_decode.rs` or `benches/cri_decode/main.rs`. Please specify bench.path if you want to use a non-default path.
error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/_fast.cpython-312-darwin.so'` failed 
with code 101

Python version information

3.12

Gufo Ping version information

0.5.0

Operation system version

macOS Sonoma 14.3.1

@nielsvanhooy nielsvanhooy added the bug Something isn't working label Feb 27, 2024
@dvolodin7
Copy link
Contributor

I'll try to repack sdiist along with benchmarks

@dvolodin7
Copy link
Contributor

Try to install 0.5.1

@nielsvanhooy
Copy link
Author

nielsvanhooy commented Feb 28, 2024

I think this got a bit further than before. but eventually it errors out.

just to be sure I added my rust and cargo versions:

cargo --version
cargo 1.76.0 (c84b36747 2024-01-18)

rustc --version
rustc 1.76.0 (07dca489a 2024-02-04)

raise build_error(e) from None
pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
32 | m.add_class::socket::GetBulkIter()?;
| ^^^^^^^^^^^ not found in socket

Full error log
`pdm add gufo-snmp

Adding packages to default dependencies: gufo-snmp
🔒 Lock successful
Changes are written to pyproject.toml.
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove

✖ Install gufo-snmp 0.5.1 failed
Retry failed jobs
✖ Install gufo-snmp 0.5.1 failed

ERRORS:
add gufo-snmp failed:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 285, in install_candidate
self.manager.install(can)
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/manager.py", line 34, in install
dist_info = installer(str(prepared.build()), self.environment, prepared.direct_url())
^^^^^^^^^^^^^^^^
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/models/candidates.py", line 418, in build
self.wheel = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/wheel.py", line 28, in build
filename = self._hook.build_wheel(out_dir, config_settings, metadata_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 209, in build_wheel
return self._call_hook('build_wheel', {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 311, in _call_hook
self._subprocess_runner(
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 261, in subprocess_runner
return log_subprocessor(cmd, cwd, extra_environ=env)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nielsvanhooij/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 107, in log_subprocessor
raise build_error(e) from None
pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
32 | m.add_class::socket::GetBulkIter()?;
| ^^^^^^^^^^^ not found in socket

Some errors have detailed explanations: E0412, E0432, E0583.
For more information about an error, try rustc --explain E0412.
error: could not compile gufo_snmp (lib) due to 11 previous errors

Caused by:
process didn't exit successfully: `/Users/nielsvanhooij/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc --crate-name gufo_snmp --edition=2021
src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type cdylib --emit=dep-info,link -C opt-level=3 -C
embed-bitcode=no -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/fast.cpython-312-darwin.so' -C metadata=3885b32172c50589 --out-dir
/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo
/target/release/deps -C strip=debuginfo -L
dependency=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps --extern
aes=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libaes-04c2a62914f2f2a8.rlib --extern
cbc=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libcbc-cf181ec2829f469d.rlib --extern
cfb_mode=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libcfb_mode-f2dccb9eee8d7db0.rlib --extern
cipher=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libcipher-911ef2a9383e1738.rlib --extern
des=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libdes-4f976b491f135e64.rlib --extern
digest=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libdigest-32e2a156418b2618.rlib --extern
enum_dispatch=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libenum_dispatch-6496fb0f11b04ba5.dylib --extern
md5=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libmd5-b6c93551a4ade8ba.rlib --extern
nom=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libnom-b03235c607168d28.rlib --extern
pyo3=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libpyo3-006f8d34e193f468.rlib --extern
rand=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/librand-11a61031ed67c63e.rlib --extern
sha1=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libsha1-9129eea9a8c68c04.rlib --extern

socket2=/private/var/folders/sn/fzdv5bys51sgwdq6mp2343jh0000gn/T/pdm-build-iwu8ovo_/target/release/deps/libsocket2-c13308f9e84f24e8.rlib(exit status: 1) error:cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --
-C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/_fast.cpython-312-darwin.so'failed with code 101

@dvolodin7
Copy link
Contributor

Can you please do checkout and run cargo check from the root of the repository?

@nielsvanhooy
Copy link
Author

nielsvanhooy commented Feb 29, 2024

sure here is the output!

cargo check
Updating crates.io index
Downloaded syn v2.0.52
Downloaded 1 crate (254.7 KB) in 0.56s
Compiling typenum v1.17.0
Compiling version_check v0.9.4
Compiling libc v0.2.153
Compiling target-lexicon v0.12.14
Compiling once_cell v1.19.0
Checking cfg-if v1.0.0
Compiling proc-macro2 v1.0.78
Compiling autocfg v1.1.0
Compiling unicode-ident v1.0.12
Compiling parking_lot_core v0.9.9
Checking smallvec v1.13.1
Compiling heck v0.4.1
Checking scopeguard v1.2.0
Compiling portable-atomic v1.6.0
Checking ppv-lite86 v0.2.17
Checking unindent v0.2.3
Compiling indoc v2.0.4
Checking memchr v2.7.1
Checking minimal-lexical v0.2.1
Compiling lock_api v0.4.11
Compiling memoffset v0.9.0
Compiling generic-array v0.14.7
Checking nom v7.1.3
Compiling quote v1.0.35
Compiling syn v2.0.52
Checking getrandom v0.2.12
Checking cpufeatures v0.2.12
Checking socket2 v0.5.6
Checking rand_core v0.6.4
Checking parking_lot v0.12.1
Checking rand_chacha v0.3.1
Checking rand v0.8.5
Compiling pyo3-build-config v0.20.3
Compiling pyo3-macros-backend v0.20.3
Checking crypto-common v0.1.6
Checking block-padding v0.3.3
Checking block-buffer v0.10.4
Checking inout v0.1.3
Checking digest v0.10.7
Compiling pyo3-ffi v0.20.3
Compiling pyo3 v0.20.3
Compiling enum_dispatch v0.3.12
Checking cipher v0.4.4
Checking md-5 v0.10.6
Checking sha1 v0.10.6
Checking aes v0.8.4
Checking cfb-mode v0.8.2
Checking cbc v0.1.2
Checking des v0.8.1
Compiling pyo3-macros v0.20.3
Checking gufo_snmp v0.5.1 (/Users/nielsvanhooij/git/new_bifrost_repos/gufo_snmp)
Finished dev [unoptimized + debuginfo] target(s) in 7.01s

that seems to be ok (not 100% sure. never worked with rust).

@dvolodin7
Copy link
Contributor

Try from root folder of the repo:

python3 -m pip install --editable .

@nielsvanhooy
Copy link
Author

sorry that it took a bit long to get back to you.
but this seems to work.

what I did was cloned this repo.

activated the venv from my own project.
and then run the command you gave me:

python3 -m pip install --editable .

this did the trick to get it working on M3 chip!

@dvolodin7
Copy link
Contributor

So I believe the problem may be related with pdm.

Anyway, we should build darwin binary packages.

@wangxin688
Copy link

So I believe the problem may be related with pdm.

Anyway, we should build darwin binary packages.

any progress on this?

@baltimorestrings
Copy link

baltimorestrings commented Aug 2, 2024

@dvolodin7, this is not just a PDB bug. We're looking at shifting an internal network monitoring tool to use this - it builds (and works excellently) on rhel7/8, but our mac dev machines require the pip install -e . hack. I'm happy to post logs confirming if you'd like. We've tried it in python3.{8,9,10,11}.

It's a bit inconvenient, but I appreciate you sharing the --edit hack so we can start work, but it would be amazing if this could be fixed. Is this something we could help with? Re: "we should build darwin binary packages"?

Our team would probably be willing to help and have plenty of darwin machines to do so with.

Thank you for the excellent library - we historically use easysnmp for its C interface to snmptools, as our needs stressed out all the python native solutions. Your library is one of the first that's fast enough to consider getting off easysnmp/easysnmp2.

@baltimorestrings
Copy link

baltimorestrings commented Aug 9, 2024

@dvolodin7, here is a recreation of the bug from scratch with just vanilla pip:

[TTW77NJH30]:~> python3.11 -m venv gufo_snmp && gufo_snmp/bin/python3 -m pip install -qU pip
[TTW77NJH30]:~> gufo_snmp/bin/pip install -q gufo_snmp
  error: subprocess-exited-with-error

  × Building wheel for gufo_snmp (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [285 lines of output]
      /private/var/folders/_z/k7431bwn657953bcnjhsnkw40000gq/T/pip-build-env-w4dhz84s/overlay/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:70: _ExperimentalConfiguration: `[tool.distutils]` in `pyproject.toml` is still *experimental* and likely to change in future releases.
        config = read_configuration(filepath, True, ignore_option_errors, dist)
      running bdist_wheel
      running build
  <... output removed ... for full output: https://gist.github.com/baltimorestrings/ea520296edf64b9d77daac741db8e617> 


      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/_fast.cpython-311-darwin.so'` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for gufo_snmp
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (gufo_snmp)

Full output: https://gist.github.com/baltimorestrings/ea520296edf64b9d77daac741db8e617

@baltimorestrings
Copy link

baltimorestrings commented Aug 9, 2024

This install works perfectly in editable mode, so we've put custom lines in our tox to recerate the following:

[TTW77NJH30]:~> gufo_snmp/bin/pip install -e git+https://github.com/gufolabs/gufo_snmp.git\#egg=gufo_snmp
Looking in indexes: https://edge.artifactory.ouroath.com:4443/artifactory/api/pypi/pypi-mirror/simple
Obtaining gufo_snmp from git+https://github.com/gufolabs/gufo_snmp.git#egg=gufo_snmp
  Cloning https://github.com/gufolabs/gufo_snmp.git to ./gufo_snmp/src/gufo-snmp
  Running command git clone --filter=blob:none --quiet https://github.com/gufolabs/gufo_snmp.git /Users/afrankel02/gufo_snmp/src/gufo-snmp
  Resolved https://github.com/gufolabs/gufo_snmp.git to commit e410cfca2267fbed6236c77e1102c31375b846b5
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: gufo_snmp
  Building editable for gufo_snmp (pyproject.toml) ... done
  Created wheel for gufo_snmp: filename=gufo_snmp-0.5.2-0.editable-cp311-cp311-macosx_13_0_arm64.whl size=4478 sha256=3566f6c512120e975a086e73bbefa00a8874dd23f5b8fd2690c8d57207997dd9
  Stored in directory: /private/var/folders/_z/k7431bwn657953bcnjhsnkw40000gq/T/pip-ephem-wheel-cache-qr8fgm0x/wheels/ce/fb/ef/889517abbb7c9b065a117ec32298a72225c81ad4e9cfb97bb4
Successfully built gufo_snmp
Installing collected packages: gufo_snmp
Successfully installed gufo_snmp-0.5.2

Since we have a workaround, this isn't too huge a deal, but I wanted to make sure you were updated as the comment above sounds like this was dismissed as PDM specific, when this bug is valid for MacOS as a whole.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants