-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Could we also get the Makefile updated to build a libfalcon.so #64
Comments
It's quite an irony that I've always been a big fan of Makefiles as opposed to configure/cmake and now it's all focused on cmake. Regarding merging, I don't see it happening any time soon. Stopwords, deep changes into cuda memory management and calculation, 16 bit cublas, a more complex kvcache, different instruction mode, a new tokenizer, finetuning detection, system prompt support, etc. you can't just integrate that into llama.cpp without a ton of work. Also many of the changes likely conflict with the vision of the core developers of the respective llama modules. Both projects have a different focus, llama.cpp is more backend ggml oriented while ggllm.cpp is aiming to put falcon into best possible realworld use as soon as possible. |
To get closer to a goal of a "ggllm.cpp" the design of a new and "final" ggml file format has been started on llama.cpp. To get closer to a goal of a "ggllm.cpp" the design of a new and "final" ggml file format has been started on llama.cpp. |
Regarding the new format, there won't be any benefits adopting it right away. Sticking with GGCC also won't remove any new features coming in from ggml. The name choice of ggllm was a mistake, I'm just sticking to it for now due to the lack of a great new name. |
Still would be nice when everything starts getting merged into llama, for one place to maintain.. I just don't like touch the C files and embarassing my self, but should be able to make it a drop in replacement for libllama.so, correct? Just thinking in terms of projects like llama_cpp
The text was updated successfully, but these errors were encountered: