This is an NVIDIA AI Workbench project to deploy LLaMA-Factory.
Please follow the LLM fine-tuning tutorial for RTX AI Toolkit here.
Minimum recommended Workbench ver: 0.50.16
Docker Desktop ver. 4.31+
- Fine-tune an LLM using Llama factory.
- Fuse generated LoRA adapter with the base model.
- Apply Post Training Quantization to your model.
- Export model to TensorRT-LLM checkpoint.
- Support for unsloth.