Skip to content

Latest commit

 

History

History
executable file
·
119 lines (103 loc) · 14 KB

README.md

File metadata and controls

executable file
·
119 lines (103 loc) · 14 KB

Run Intel® AI Reference Models in a Jupyter Notebook

This Jupyter notebook helps you choose and run a comparison between two models from the Intel® AI Reference Models repo using Intel® Optimizations for TensorFlow*. When you run the notebook, it installs required package dependencies, displays information about your platform, lets you choose the two models to compare, runs those models, and finally displays a performance comparison chart.

Supported Models

Intel® Data Center CPU Workloads

Model Framework Mode Supported Precisions
ResNet 50v1.5 Sapphire Rapids TensorFlow Inference Int8 FP32 BFloat16 BFloat32
ResNet 50v1.5 Sapphire Rapids TensorFlow Training FP32 BFloat16 BFloat32
ResNet 50 PyTorch Inference Int8 FP32 BFloat16 BFloat32
ResNet 50 PyTorch Training FP32 BFloat16 BFloat32
Vision Transformer PyTorch Inference FP32 BFloat16 BFloat32 FP16 INT8
3D U-Net TensorFlow Inference FP32 BFloat16 Int8
BERT large Sapphire Rapids Tensorflow Inference FP32 BFloat16 Int8 BFloat32
BERT large Sapphire Rapids Tensorflow Training FP32 BFloat16 BFloat32
BERT large (Hugging Face) TensorFlow Inference FP32 FP16 BFloat16 BFloat32
BERT large PyTorch Inference FP32 Int8 BFloat16 BFloat32
BERT large PyTorch Training FP32 BFloat16 BFloat32
DistilBERT base PyTorch Inference FP32 BF32 BF16Int8-FP32 Int8-BFloat16 BFloat32
RNN-T PyTorch Inference FP32 BFloat16 BFloat32
RNN-T PyTorch Training FP32 BFloat16 BFloat32
GPTJ 6B PyTorch Inference FP32 FP16 BFloat16 BF32 INT8
GPTJ 6B MLPerf PyTorch Inference INT4
ChatGLMv3 6B PyTorch Inference FP32 FP16 BFloat16 BF32 INT8
BERT TensorFlow Inference FP32
Mask R-CNN PyTorch Inference FP32 BFloat16 BFloat32
Mask R-CNN PyTorch Training FP32 BFloat16 BFloat32
SSD-ResNet34 PyTorch Inference FP32 Int8 BFloat16 BFloat32
SSD-ResNet34 PyTorch Training FP32 BFloat16 BFloat32
Yolo V7 PyTorch Inference Int8 FP32 FP16 BFloat16 BFloat32
Wide & Deep TensorFlow Inference FP32
DLRM PyTorch Inference FP32 Int8 BFloat16 BFloat32
DLRM PyTorch Training FP32 BFloat16 BFloat32
DLRM v2 PyTorch Inference FP32 FP16 BFloat16 BFloat32 Int8
Stable Diffusion TensorFlow Inference FP32 BFloat16 FP16
Stable Diffusion PyTorch Inference FP32 BFloat16 FP16 BFloat32 Int8-FP32 Int8-BFloat16
Stable Diffusion PyTorch Training FP32 BFloat16 FP16 BFloat32
Latent Consistency Models(LCM) PyTorch Inference FP32 BFloat16 FP16 BFloat32 Int8-FP32 Int8-BFloat16
GraphSAGE TensorFlow Inference FP32 BFloat16 FP16 Int8 BFloat32

Intel® Data Center CPU Workloads

Model Framework Mode Platform Supported Precisions
ResNet 50v1.5 TensorFlow Inference Flex Series Float32 TF32 Float16 BFloat16 Int8
ResNet 50 v1.5 TensorFlow Training Max Series BFloat16 FP32
ResNet 50 v1.5 PyTorch Inference Flex Series, Max Series, Arc Series Int8 FP32 FP16 TF32
ResNet 50 v1.5 PyTorch Training Max Series, Arc Series BFloat16 TF32 FP32
DistilBERT PyTorch Inference Flex Series, Max Series FP32 FP16 BF16 TF32
DLRM v1 PyTorch Inference Flex Series FP16 FP32
SSD-MobileNet* PyTorch Inference Arc Series INT8 FP16 FP32
EfficientNet PyTorch Inference Flex Series FP16 BF16 FP32
EfficientNet TensorFlow Inference Flex Series FP16
FBNet PyTorch Inference Flex Series FP16 BF16 FP32
Wide Deep Large Dataset TensorFlow Inference Flex Series FP16
YOLO V5 PyTorch Inference Flex Series FP16
BERT large PyTorch Inference Max Series, Arc Series BFloat16 FP32 FP16
BERT large PyTorch Training Max Series, Arc Series BFloat16 FP32 TF32
BERT large TensorFlow Training Max Series BFloat16 TF32 FP32
DLRM v2 PyTorch Inference Max Series FP32 BF16
DLRM v2 PyTorch Training Max Series FP32 TF32 BF16
3D-Unet PyTorch Inference Max Series FP16 INT8 FP32
3D-Unet TensorFlow Training Max Series BFloat16 FP32
Stable Diffusion PyTorch Inference Flex Series, Max Series, Arc Series FP16 FP32
Stable Diffusion TensorFlow Inference Flex Series FP16 FP32
Mask R-CNN TensorFlow Inference Flex Series FP32 Float16
Mask R-CNN TensorFlow Training Max Series FP32 BFloat16
Swin Transformer PyTorch Inference Flex Series FP16
FastPitch PyTorch Inference Flex Series FP16
UNet++ PyTorch Inference Flex Series FP16
RNN-T PyTorch Inference Max Series FP16 BF16 FP32
RNN-T PyTorch Training Max Series FP32 BF16 TF32
IFRNet PyTorch Inference Flex Series FP16
RIFE PyTorch Inference Flex Series FP16

Environment Setup

Instead of installing or updating packages system-wide, it's a good idea to install project-specific Python packages in a Python virtual environment localized to your project. The Python virtualenv package lets you do just that. Using virtualenv is optional, but recommended.

The jupyter notebook runs on Ubuntu distribution for Linux.

  1. Virtualenv Python Environment Install virtualenv on Ubuntu using these commands:

    sudo apt-get update
sudo apt-get install python-dev python-pip
sudo pip install -U virtualenv  # system-wide install

    Activate virtual environment using the following command:

    virtualenv -p python ai_ref_models
source ai_ref_models/bin/activate

  2. Jupyter Notebook Support:

    Install Jupyter notebook support with the command:

       pip install notebook

    Refer to the Installing Jupyter guide for details.

How to Run the Notebook

  1. Clone the Intel® AI Reference Models repo: 
    git clone https://github.com/IntelAI/models.git
  2. Launch the Jupyter notebook server: jupyter notebook --ip=0.0.0.0
  3. Follow the instructions to open the URL with the token in your browser, something like this: http://127.0.0.1:8888/tree?token=<token>
  4. Browse to the models/notebooks/ folder
  5. Click the AI_Reference_Models notebook file - AI_Reference_Models.ipynb.
  6. Read the instructions and run through each notebook cell, in order, ending with a display of the analysis results. Note that some cells prompt you for input, such as selecting the model number you'd like to run.
  7. When done, you should deactivate the virtualenv, if you used one, with the command: deactivate