-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
can't start new thread #1624
Comments
It seems like that script tries to use 8 threads to write the vocabulary. If you have some sort of resource limits set for your user or with Docker that could be limiting the number of threads, you could try adjusting that (no suggestions, I don't use Docker or Windows). If you can figure out how to edit the Line 967 in a670464
You could try setting the concurrency to a lower value there. Unfortunately, it doesn't appear configurable without actually editing the script. This may or may not help you: https://stackoverflow.com/questions/47490307/editing-files-inside-of-a-docker-container Looks like the scripts and such will be under the |
@KerfuffleV2 I changed the concurrency to 1, committed a new image and run again
but I got the same problem:
Does anyone run the llama.cpp correct in Windows 7? |
Hmm, seems like starting threads just doesn't work with your setup. I don't know if it's an issue with the Docker container or something else. Sorry, I also don't know about the other question. I would guess there are very few people still on Windows 7 at this point so there is a decent chance that particular setup isn't very well tested. One thing you can possibly try is changing that parallel map to just a normal map. I think you could do: ndarrays = map(do_item, model.items()) Making sure to preserve the existing indentation when you change it. |
@KerfuffleV2 Thanks that works, I change bounded_parallel_map into map, and I get ggml-model-f16.bin
when I execute:
and I checked my quantize file:
But the architecture of docker image named ghcr.io/ggerganov/llama.cpp:full is amd64:
and my environment is:
SO I'M CONFUSED: |
Unfortunately, I think we're reaching the point where I can't really help you anymore. I haven't actually quantized my own models or used the container, and I also don't use Windows. So I'm just going by random stuff I've seen. There are two possible explanations I can think of here:
In the case of #2, I'm not sure there's a lot you can do other than find the model already converted/quantized (it's usually not that hard, I can't tell you specifically since directly linking to those models isn't allowed) and simply download it or try to compile the project locally yourself. I don't know if WSL even works with Windows 7, so I can't give you advice on that part. |
@KerfuffleV2 thanks for your help, the CPU I used supports AVX2, and docker architecture is amd64, but type of quntize file is x86_64, and I have tested , not only the quantization phase, but in the run phase (download ggml_model_q4_0.bin from huggingface) I get the same question, that means that the distance between success if only this question. |
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
I just wanna running ggml-model-q4_0.bin on my Windows 7.
Current Behavior
I followed the instruction written in README.md and executed :
but I got the error:
Environment and Context
Docker Toolbox 1.13.1
docker client: 1.13.1 os/arch: windows 7 /amd64
docker server:19.03.12 os/arch:ubuntu 22.04 /amd64
Steps to Reproduce
What should I do ? Thanks in advance
The text was updated successfully, but these errors were encountered: