Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It still does not work with some GGML v3 models #11

Open
idobric opened this issue May 26, 2023 · 5 comments
Open

It still does not work with some GGML v3 models #11

idobric opened this issue May 26, 2023 · 5 comments

Comments

@idobric
Copy link

idobric commented May 26, 2023

For example:
https://huggingface.co/TheBloke/gpt4-alpaca-lora-30B-4bit-GGML/tree/main
The main branch should have GGML v3 but python crashes when trying to run this model.
I can run it using llama_cpp package.

Thanks

@absadiki
Copy link
Owner

@idobric, it seems like there are new updates again 😅 the main branch is pointing to the GGML v2 I guess.
I will update it to the latest and let you know.

@absadiki
Copy link
Owner

@idobric, I updated llama.cpp to the latest release and pyllamacpp is working now with the model you provided above.
I've pushed a new 2.4.0 version to PYPI. Could you please test it on your end ?

@ParisNeo
Copy link
Contributor

Hi and thank you very much for doing this.

What's really hard about all these breaking changes is that they never keep retrocompatibility whith previous models versions.

Your bindings are supported at my gpt4all-ui. It would be cool if you test it and upgrade the binding part as you change yours your bindings has -a at the end of the name (for abdeladim) as i also support other developers bindings. Marella has also made some good bindings and i add -m to distinguish his from yours. I probably gonna change the bindings list to have a card that shows who built it and give a link to their repos to show some love.

I have created a binding template to show how to interface any new binding with the app. It is very easy, just implement a class and you're good to go.

I finally built a video to show how the basic use of the ui. I hope you like it:
https://www.youtube.com/watch?v=ds_U0TDzbzI&ab_channel=ParisNeo

@absadiki
Copy link
Owner

@ParisNeo, yeah, that's the big problem, it's hard to know which version for which model, especially with the myriad of ggml models out there.

So finally the 2.0 release of gpt4all-ui is here, it looks fantastic 👏

Yeah, I noticed that you support my bindings. Glad you found them useful.
I didn't change anything to the API so basically everything should still work I guess. If I did I'll try to submit a PR.

Keep up the good work 💪

@ParisNeo
Copy link
Contributor

Thanks man :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants