How do i use convert-unversioned-ggml-to-ggml.py?

[Shreyas-ITB]

Hi it told me to use the convert-unversioned-ggml-to-ggml.py file and gave me an error saying your gpt4all model is too old. So i converted the gpt4all-lora-unfiltered-quantized.bin file with llama tokenizer. And it generated some kind of orig file in the same directory where the model was. When i tried to run the miku.sh file which had the latest generated file as model it gave me another error stating this
main: seed = 1680783525 llama_model_load: loading model from './models/gpt4all-7B/gpt4all-lora-unfiltered-quantized.bin' - please wait ... ./models/gpt4all-7B/gpt4all-lora-unfiltered-quantized.bin: invalid model file (bad magic [got 0x67676d66 want 0x67676a74]) you most likely need to regenerate your ggml files the benefit is you'll get 10-100x faster load times see https://github.com/ggerganov/llama.cpp/issues/91 use convert-pth-to-ggml.py to regenerate from original pth use migrate-ggml-2023-03-30-pr613.py if you deleted originals llama_init_from_file: failed to load model main: error: failed to load model './models/gpt4all-7B/gpt4all-lora-unfiltered-quantized.bin'

How do i use the conversion? did i do something wrong?

[Showgofar]

the same here

python3 convert-unversioned-ggml-to-ggml.py models/ggml-alpaca-7b-q4.bin models/ggml-alpaca-7b-q4-new.bin

Traceback (most recent call last):
File "convert-unversioned-ggml-to-ggml.py", line 100, in
main()
File "convert-unversioned-ggml-to-ggml.py", line 94, in main
tokenizer = SentencePieceProcessor(args.tokenizer_model)
File "/usr/local/lib64/python3.8/site-packages/sentencepiece/init.py", line 447, in Init
self.Load(model_file=model_file, model_proto=model_proto)
File "/usr/local/lib64/python3.8/site-packages/sentencepiece/init.py", line 905, in Load
return self.LoadFromFile(model_file)
File "/usr/local/lib64/python3.8/site-packages/sentencepiece/init.py", line 310, in LoadFromFile
return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
OSError: Not found: "models/ggml-alpaca-7b-q4-new.bin": No such file or directory Error #2

python3 migrate-ggml-2023-03-30-pr613.py models/7B/ggml-alpaca-7b-q4.bin models/7B/ggml-alpaca-7b-q4-new.bin

Traceback (most recent call last):
File "migrate-ggml-2023-03-30-pr613.py", line 311, in
main()
File "migrate-ggml-2023-03-30-pr613.py", line 272, in main
tokens = read_tokens(fin, hparams)
File "migrate-ggml-2023-03-30-pr613.py", line 133, in read_tokens
word = fin.read(length)
ValueError: read length must be non-negative or -1

also maybe its can be already converted model somewhere exist?

[Naugustogi]

same ValueError: read length must be non-negative or -1

[HCBlackFox]

same ValueError: read length must be non-negative or -1

+1

[Showgofar]

I found out that problem was that you need numpy module

[Triptolemus]

@Shreyas-ITB I had the same confusion as you, since the readme doesn't explain all the conversion steps, but the error response you received does identify the next step to follow. Run the suggested script, for example:

python migrate-ggml-2023-03-30-pr613.py models/alpaca/ggml-alpaca-7b-q4.bin models/alpaca/ggml-alpaca-7b-q4-new.bin

Then you can use the converted model, for example:

./main -m ./models/alpaca/ggml-alpaca-7b-q4-new.bin -n 256 --repeat_penalty 1.0 --color -i -r "User: " -f prompts/chat-with-bob.txt

[vinitran]

@Triptolemus i just run your script but it has 1 error, do you have any solution ?
Traceback (most recent call last):
File "/home/vinitran/Documents/Ai/go-llama.cpp/llama.cpp/migrate-ggml-2023-03-30-pr613.py", line 311, in
main()
File "/home/vinitran/Documents/Ai/go-llama.cpp/llama.cpp/migrate-ggml-2023-03-30-pr613.py", line 272, in main
tokens = read_tokens(fin, hparams)
File "/home/vinitran/Documents/Ai/go-llama.cpp/llama.cpp/migrate-ggml-2023-03-30-pr613.py", line 133, in read_tokens
word = fin.read(length)
ValueError: read length must be non-negative or -1

[IhorChornyi314]

For me the problem was that I ran migrate-ggml-2023-03-30-pr613.py on an old model without running convert-unversioned-ggml-to-ggml.py on it first. Once I did that everything worked fine

[Triptolemus]

@vinitran Sorry, my previous comment had the wrong model names in the suggested command, but I've edited it to correct that. To confirm, did you first run the convert-unversioned-ggml-to-ggml.py script, then run the migrate-ggml-2023-03-30-pr613.py script on the model outputted as a result of the first script?

[IhorChornyi314]

Yup

[chofercrd]

the same here

python3 convert-unversioned-ggml-to-ggml.py models/ggml-alpaca-7b-q4.bin models/ggml-alpaca-7b-q4-new.bin

Traceback (most recent call last): File "convert-unversioned-ggml-to-ggml.py", line 100, in main() File "convert-unversioned-ggml-to-ggml.py", line 94, in main tokenizer = SentencePieceProcessor(args.tokenizer_model) File "/usr/local/lib64/python3.8/site-packages/sentencepiece/init.py", line 447, in Init self.Load(model_file=model_file, model_proto=model_proto) File "/usr/local/lib64/python3.8/site-packages/sentencepiece/init.py", line 905, in Load return self.LoadFromFile(model_file) File "/usr/local/lib64/python3.8/site-packages/sentencepiece/init.py", line 310, in LoadFromFile return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg) OSError: Not found: "models/ggml-alpaca-7b-q4-new.bin": No such file or directory Error #2

python3 migrate-ggml-2023-03-30-pr613.py models/7B/ggml-alpaca-7b-q4.bin models/7B/ggml-alpaca-7b-q4-new.bin

Traceback (most recent call last): File "migrate-ggml-2023-03-30-pr613.py", line 311, in main() File "migrate-ggml-2023-03-30-pr613.py", line 272, in main tokens = read_tokens(fin, hparams) File "migrate-ggml-2023-03-30-pr613.py", line 133, in read_tokens word = fin.read(length) ValueError: read length must be non-negative or -1

also maybe its can be already converted model somewhere exist?

python3 convert-unversioned-ggml-to-ggml.py /models ggml-alpaca-q4.bin
a mi me da out of memory pero entiendo que funciona

[chofercrd]

python3 convert-unversioned-ggml-to-ggml.py models/ ggml-alpaca-q4.bin
notar el cambio en el directorio models/

[prusnak]

try the new convert.py script that is now in master