Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could we also get the Makefile updated to build a libfalcon.so #64

Open
linuxmagic-mp opened this issue Jul 14, 2023 · 3 comments
Open

Comments

@linuxmagic-mp
Copy link

Still would be nice when everything starts getting merged into llama, for one place to maintain.. I just don't like touch the C files and embarassing my self, but should be able to make it a drop in replacement for libllama.so, correct? Just thinking in terms of projects like llama_cpp

@linuxmagic-mp linuxmagic-mp changed the title Could we also get the Makefule updated to build a libfalcon.so Could we also get the Makefile updated to build a libfalcon.so Jul 14, 2023
@cmp-nct
Copy link
Owner

cmp-nct commented Jul 14, 2023

It's quite an irony that I've always been a big fan of Makefiles as opposed to configure/cmake and now it's all focused on cmake.
I've pushed an update to add libfalcon.so, should work.

Regarding merging, I don't see it happening any time soon.
The changes in ggllm.cpp (which probably will be renamed into falcon centric at a later time) are a lot deeper than just an evaluation routine. I'm quite sure llama.cpp will show basic support for falcon soon but it won't include the same features and improvements we have here.
"ggllm" was originally chosen because that is what I wanted llama.cpp to become, it was meant as a "lobbying fork". By now we have deviated too far from that original intention.

Stopwords, deep changes into cuda memory management and calculation, 16 bit cublas, a more complex kvcache, different instruction mode, a new tokenizer, finetuning detection, system prompt support, etc. you can't just integrate that into llama.cpp without a ton of work. Also many of the changes likely conflict with the vision of the core developers of the respective llama modules.

Both projects have a different focus, llama.cpp is more backend ggml oriented while ggllm.cpp is aiming to put falcon into best possible realworld use as soon as possible.

@maddes8cht
Copy link

To get closer to a goal of a "ggllm.cpp" the design of a new and "final" ggml file format has been started on llama.cpp.
Will "your" ggllm.cpp adopt this new file format, or rather not?
I think there will be more and more functions in ggml that will refer to the new format?

To get closer to a goal of a "ggllm.cpp" the design of a new and "final" ggml file format has been started on llama.cpp.
Will "your" ggllm.cpp adopt this new file format, or rather not?
I think there will be more and more functions in ggml that will refer to the new format?
The new file format could eventually become the common denominator from which a common base is built, which then splits more specifically into the functions of individual models.

@cmp-nct
Copy link
Owner

cmp-nct commented Jul 15, 2023

Regarding the new format, there won't be any benefits adopting it right away. Sticking with GGCC also won't remove any new features coming in from ggml.
If the new format turns out to flexible enough I think it makes sense going into that direction but as a lower priority development. It's quite a bit of work and my focus for now is performance and new features.

The name choice of ggllm was a mistake, I'm just sticking to it for now due to the lack of a great new name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants