Bloom model integration

[Miezhiko]

maybe

there it is https://huggingface.co/bigscience/bloom

[guillaume-be]

Hello @Miezhiko ,

Thank you for the suggestion - I will look into it in the next few weeks. I will probably target the smaller models for an initial implementation as I don't have access to a machine which can load the full size model at the moment.

[danforbes]

What is the status of this effort?

[guillaume-be]

Hello @danforbes ,
I haven't had the chance to look into this yet, but would have some capacity starting next week. Is that something you would like to look into?

[danforbes]

Honestly, I would love to, but I'm afraid I would need a lot of help and may not be the best person for the job - I'm not a terribly experienced Rust developer, nor do I really understand all that much about LLMs 😅 At this point I would mostly describe myself as an enthusiast that happens to have a fair amount of programming experience 🤓 I recently authored this PR on the rustformers/llama-rs repository, which was a bit of a basic refactor and a lot of copy-pasta 😅 It did get me wondering if there are opportunities for that project to have some type of interaction with this one, though 🤔 That PR also has a fair amount of code related to BLOOM inference, so it may be helpful in integrating BLOOM with this project. Let me know if you have any thoughts on the above.