Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llama.cpp: add model_converters variant #27526

Open
wants to merge 47 commits into
base: master
Choose a base branch
from

Conversation

i0ntempest
Copy link
Member

Description

Type(s)
  • bugfix
  • enhancement
  • security fix
Tested on

macOS x.y
Xcode x.y / Command Line Tools x.y.z

Verification

Have you

  • followed our Commit Message Guidelines?
  • squashed and minimized your commits?
  • checked that there aren't other open pull requests for the same change?
  • referenced existing tickets on Trac with full URL in commit message?
  • checked your Portfile with port lint?
  • tried existing tests with sudo port test?
  • tried a full install with sudo port -vst install?
  • tested basic functionality of all binary files?
  • checked that the Portfile's most important variants haven't been broken?

@macportsbot
Copy link

Notifying maintainers:
@blair for port py-tensorflow_estimator.
@mdouze for port libfaiss.
@beauby for port libfaiss.
@emcrisostomo for port py-absl, py-tensorboard.
@cjones051073 for port py-blis, py-confection, py-plac, py-preshed, py-srsly, py-thinc, py-wasabi.
@reneeotten for port py-cookiecutter.
@stromnov for port py-scikit-learn.

@macportsbot macportsbot added type: enhancement type: submission maintainer: open Affects an openmaintainer port by: member Created by a member with commit rights labels Jan 27, 2025
@i0ntempest i0ntempest force-pushed the llama branch 3 times, most recently from 5209a36 to 409ad9a Compare January 27, 2025 17:58
@i0ntempest
Copy link
Member Author

@mascguy Please take a look!
This massive PR took me like 2 days, and there's almost certainly something that I fricked up. For one I cannot track the dependency relations of all the python ports I changed anymore.
What I've done:
Ran through all the dependencies needed for the four conversion scripts to work, adding compatibility up to Python 3.12 (or 3.13 if the package supports it), and updating them if possible.

What I've tested and works:

  1. Importing gguf, torch, tensorflow, transformers in python
  2. Running the tensorflow test script from Apple here
  3. Running all the conversion scripts without args (that is I did not test any actual conversions)

Some notes:

  1. py-preshed must be downgraded for py-spaCy to build
  2. py-tensorflow-macos was updated to use tensorflow pypi instead of tensorflow-macos, which seems defunct. This is the only way we can get to python 3.12, and I didn't dare touch the actual py-tensorflow port
  3. Multiple build workarounds needed because our numpy is ancient
  4. Old protobuf3-cpp blocked me from updating grpc

@mascguy
Copy link
Member

mascguy commented Jan 27, 2025

I'll take a closer look.

Also adding @essandess for review/comment, as he's contributed to a number of Python ports in this space.

@i0ntempest i0ntempest changed the title llama.cpp 4567, add model_converters variant llama.cpp: add model_converters variant Jan 28, 2025
@i0ntempest i0ntempest force-pushed the llama branch 2 times, most recently from ec91fe7 to 6dad512 Compare January 29, 2025 02:27
@i0ntempest
Copy link
Member Author

i0ntempest commented Jan 29, 2025

I haven’t ironed out all the problems yet but all 3 CI tests fail at 381 minutes, is this a timeout?
Also I can’t see any log from 15.0 CI.

@mascguy
Copy link
Member

mascguy commented Jan 29, 2025

I haven’t ironed out all the problems yet but all 3 CI tests fail at 381 minutes, is this a timeout?
Also I can’t see any log from 15.0 CI.

I believe so, as the timeout is 6-ish hours.

@i0ntempest
Copy link
Member Author

Right, because I cannot see any logs from macOS 15 CI I'm looking at macOS 14 test. Two problematic ports remains:

  1. py312-spaCy fails with numerous of:
2025-01-28T17:38:17.7007800Z spacy/matcher/levenshtein.c:1767:3: error: no member named 'use_tracing' in 'struct _PyCFrame'
2025-01-28T17:38:17.7008500Z  1767 |   __Pyx_TraceCall("levenshtein", __pyx_f[0], 14, 0, __PYX_ERR(0, 14, __pyx_L1_error));
2025-01-28T17:38:17.7008990Z       |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2025-01-28T17:38:17.7009480Z spacy/matcher/levenshtein.c:1216:15: note: expanded from macro '__Pyx_TraceCall'
2025-01-28T17:38:17.7009920Z  1216 |           if (__Pyx_IsTracing(tstate, 1, 1)) {\
2025-01-28T17:38:17.7010280Z       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Seems cython related. I have edited it to use cython-compat and it does build on my machine.
2. py*-torchvision fails with:

2025-01-28T17:56:17.6172010Z /usr/bin/clang -arch arm64 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -DPNG_FOUND=1 -I/opt/local/include/libpng16 -I/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/torch/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/torch/include/TH -I/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/torch/include/THC -I/opt/local/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c /opt/local/var/macports/build/_Users_runner_work_macports-ports_macports-ports_ports_python_py-torchvision/py312-torchvision/work/vision-0.20.0/torchvision/csrc/io/image/cpu/giflib/dgif_lib.c -o build/temp.macosx-14.0-arm64-cpython-312/opt/local/var/macports/build/_Users_runner_work_macports-ports_macports-ports_ports_python_py-torchvision/py312-torchvision/work/vision-0.20.0/torchvision/csrc/io/image/cpu/giflib/dgif_lib.o -g0 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -DTORCH_EXTENSION_NAME=image -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
2025-01-28T17:56:17.6361250Z error: invalid argument '-std=c++17' not allowed with 'C'
2025-01-28T17:56:17.6385170Z error: command '/usr/bin/clang' failed with exit code 1

Portfile does not mention anything related to c++17, and again it does build on my system. I've made a bit more changes to it, let's see if it passes this time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
by: member Created by a member with commit rights maintainer: open Affects an openmaintainer port type: enhancement type: submission type: update
Development

Successfully merging this pull request may close these issues.

8 participants