[LLM] Adds Llama 3.1 reference implementation code #781
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
| FROM ${NEMO_BASE_IMAGE} AS nemo-base-image | ||
|
|
||
| RUN pip uninstall transformers -y | ||
| RUN pip install transformers blobfile |
There was a problem hiding this comment.
Please freeze the versions for reproducibility
|
|
||
| RUN pip uninstall transformers -y | ||
| RUN pip install transformers blobfile | ||
| RUN pip install prettytable |
|
|
||
| ### Steps to run and time | ||
|
|
||
| To train Llama 3.1 405B, we need to fill out all fields in [config.sh](./config.sh). This file contains all configurations for Slurm cluster access and job submission configurations, directory mappings, containers, and model configurations. |
There was a problem hiding this comment.
Can we have an example config.sh file with all these values filled out so users know what the paths should point to or what the contents for these values should look like? For example mention what the contents of the tokenizer path should look like and where it should be saved. Maybe also state that these paths need to be mounted (if that's necessary) during docker run.
|
|
||
| We use Mixtral 8x22B tokenizer in this benchmark. Tokenizer files can be downloaded [here](https://huggingface.co/mistralai/Mixtral-8x22B-v0.1/tree/main). Only the five files containing tokenizer-related contents (`special_tokens_map.json`, `tokenizer.json`, `tokenizer.model`, `tokenizer.model.v1`, `tokenizer_config.json`) are needed. | ||
|
|
||
| ### Data preprocessing |
There was a problem hiding this comment.
Once the processed data is uploaded to MLC, please move these sections to an appendix and just share instructions on how to download the processed data.
|
|
||
| exp_name = size | ||
|
|
||
| if size == "8b": |
There was a problem hiding this comment.
Mention that 8b and 70b are there only for debugging and that the benchmark is actually llama3.1 405b to avoid confusion
| model_group.add_argument( | ||
| "--size", | ||
| type=str, | ||
| default="8b", |
There was a problem hiding this comment.
Change default to 405b
Adds the initial Llama 3.1 reference implementation.