LLM Insights

1. Infrastructure/Platform
   • Hardware
   • Alternative Accelerators
2. Data Engineering
   • Common Crawl
3. Pre-Training Stage
   • Tokenization
   • Model Architecture
   • Multi-Node
   • Training
   • Base Model Evaluation
4. Fine-Tuning Stage
   • Adapters
5. Alignment Stage
   • Reducing Hallucination
6. Evaluation Stage
   • Reasoning
7. Inference Stage
   • Estimating Model Size
   • Speed
   • Throughput
   • Guidance
8. Applications
   • Agents

Acknowledgements

I'm grateful to my employers for trusting me to lead the team that built the GPU supercompute platform/infrastructure and to co-lead the team doing LLM pre-training. This allowed me to work on large on-premise GPU compute clusters with A100s and then H100s, which is certainly a privilege. Hopefully sharing some of these notes and insights helps the community.