Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regression: convert.py Could not find any of ['vocab.json'] for deepseek-coder models (--pad-vocab --vocab-type bpe) #5973

Closed
0x4E69676874466F78 opened this issue Mar 10, 2024 · 0 comments · Fixed by #6355

Comments

@0x4E69676874466F78
Copy link

Regression commit 4d4d236

python.exe convert.py --outtype f32 --pad-vocab --vocab-type bpe B:\deepseek-coder-7b-instruct-v1.5-qlora-amenokaku-code
...
Found vocab files: {'spm': None, 'bpe': None, 'hfft': WindowsPath('B:/deepseek-coder-7b-instruct-v1.5-qlora-amenokaku-code/tokenizer.json')}
Traceback (most recent call last):
  File "A:\gitproject\llama.cpp\convert.py", line 1479, in <module>
    main()
  File "A:\gitproject\llama.cpp\convert.py", line 1447, in main
    vocab, special_vocab = vocab_factory.load_vocab(args.vocab_type.split(","), model_parent_path)
  File "A:\gitproject\llama.cpp\convert.py", line 1323, in load_vocab
    vocab_type, path = self._select_file(vocab_types)
  File "A:\gitproject\llama.cpp\convert.py", line 1310, in _select_file
    raise FileNotFoundError(f"Could not find any of {[self._FILES[vt] for vt in vocab_types]}")
FileNotFoundError: Could not find any of ['vocab.json']

Same with the original deepseek-coder model.

git checkout aa23412 convert.py
OK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant