[New Model]: Llama 3.2

[shermansiu]

The model to consider.

• Huggingface collection: https://huggingface.co/collections/meta-llama/llama-32-66f448ffc8c32f949b04c8cf

Highlighted model weights:

• 1B Instruct Model: https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct
• 3B Instruct Model: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
• 11B Instruct Model: https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct
• 90B Instruct Model: https://huggingface.co/meta-llama/Llama-3.2-90B-Vision-Instruct

The closest model vllm already supports.

https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/llama.py
https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/llava_onevision.py

What's your difficulty of supporting the model you want?

Yes, Llama 3.2 is multimodal with a different architecture than previous multimodal Llama models.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.