Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Torch not compiled with cuda enabled #2

Open
zxstyyy opened this issue Feb 10, 2025 · 1 comment
Open

Torch not compiled with cuda enabled #2

zxstyyy opened this issue Feb 10, 2025 · 1 comment

Comments

@zxstyyy
Copy link

zxstyyy commented Feb 10, 2025

i am on a macbook air m1. installed pytorch from https://developer.apple.com/metal/pytorch/. pytorch installed successfully but when i run main.py i get the output:
[INFO] Initializing SearchSession for query_id=a973ad1e
model.safetensors.index.json: 100%|███████████████| 65.1k/65.1k [00:00<00:00, 70.1MB/s]
model-00001-of-00002.safetensors: 100%|███████████| 4.99G/4.99G [01:58<00:00, 42.2MB/s]
model-00002-of-00002.safetensors: 100%|█████████████| 862M/862M [00:20<00:00, 42.3MB/s]
Downloading shards: 100%|████████████████████████████████| 2/2 [02:19<00:00, 69.60s/it]
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/Users/danielmoss/NanoSage/main.py", line 54, in
main()
File "/Users/danielmoss/NanoSage/main.py", line 32, in main
session = SearchSession(
^^^^^^^^^^^^^^
File "/Users/danielmoss/NanoSage/search_session.py", line 174, in init
self.model, self.processor, self.model_type = load_retrieval_model(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/danielmoss/NanoSage/knowledge_base.py", line 16, in load_retrieval_model
model = ColPaliForRetrieval.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4245, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4815, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/transformers/modeling_utils.py", line 873, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/opt/anaconda3/lib/python3.11/site-packages/accelerate/utils/modeling.py", line 330, in set_module_tensor_to_device
new_value = value.to(device)
^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/torch/cuda/init.py", line 310, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

How can i enable cuda since it says its not working?

@zxstyyy
Copy link
Author

zxstyyy commented Feb 10, 2025

just realized cuda is not for mac silicon...
is there something else i can use to utilize gpu acceleration? thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant