Deep Learning Notes

This repository consists of different Deep Learning papers summaries. Ranging from Model Architecture to Acceleration.

Queue:

Low-Bit Training

• QLoRA
• ZeRO-1
• BF16 Training - Stochastic Sampling
• 8-Bit Optimizer
• 4-Bit Optimizer

Machine Learning Compilers

• Flex-Attention

Foundation Models

• DeepSeekV2
• DeepSeekV3

GPU Architecture

• NVIDIA Hopper - H100 Tensor Core GPU Architecture
• NVIDIA Ampere - A100 Tensor Core GPU Architecture
• NVIDIA Turing - TURING GPU Architecture
• NVIDIA Volta - TESLA V100 Architecture
• NVIDIA Pascal - TESLA P100 Architecture
• NVIDIA Kepler - CUDA Compute Architecture
• NVIDIA Fermi - CUDA Compute Architecture
• AMD CDNA 2 - CDNA 2 GPU Architecture
• AMD CDNA - CDNA GPU Architecture

Blogposts:

• Colfax Reasearch Papers About Acceleration