Official repo for the paper: EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge
Follow the instructions of the BabyLLaMA to implement the training environment, and BabyLM Challenge to implement the evaluation environment.
- Download dataset from BabyLM Challenge
- Clean the dataset according to BabyLLaMA
- Pretrain teacher model
- Download FP16 LLaMA-58M model from BabyLLaMA
- QAT with scripts in
distill_train/scripts/ - Evaluation with scripts in
evaluation_pipeline/
@article{shen2024edgeqat,
title={EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge},
author={Shen, Xuan and Kong, Zhenglun and Yang, Changdi and Han, Zhaoyang and Lu, Lei and Dong, Peiyan and others},
journal={arXiv preprint arXiv:2402.10787},
year={2024}
}