Skip to content

Commit

Permalink
add W8A8 support and example
Browse files Browse the repository at this point in the history
Signed-off-by: Mengni Wang <[email protected]>
  • Loading branch information
mengniwang95 committed Jun 20, 2024
1 parent 0f8c3a6 commit a4410a2
Show file tree
Hide file tree
Showing 75 changed files with 11,978 additions and 1,336 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Step-by-Step

This example load an image classification model from [ONNX Model Zoo](https://github.com/onnx/models) and confirm its accuracy and speed based on [ILSVR2012 validation Imagenet dataset](http://www.image-net.org/challenges/LSVRC/2012/downloads). You need to download this dataset yourself.

# Prerequisite

## 1. Environment

```shell
pip install onnx-neural-compressor
pip install -r requirements.txt
```

> Note: Validated ONNX Runtime [Version](/docs/source/installation_guide.md#validated-software-environment).
## 2. Prepare Model

```shell
python prepare_model.py --output_model='resnet50-v1-12.onnx'
```

## 3. Prepare Dataset

Download dataset [ILSVR2012 validation Imagenet dataset](http://www.image-net.org/challenges/LSVRC/2012/downloads).

Download label:

```shell
wget http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz
tar -xvzf caffe_ilsvrc12.tar.gz val.txt
```

# Run


## 1. Quantization

Quantize model with QLinearOps:

```bash
bash run_quant.sh --input_model=path/to/model \ # model path as *.onnx
--dataset_location=/path/to/imagenet \
--label_path=/path/to/val.txt \
--output_model=path/to/save
```

Quantize model with QDQ mode:

```bash
bash run_quant.sh --input_model=path/to/model \ # model path as *.onnx
--dataset_location=/path/to/imagenet \
--label_path=/path/to/val.txt \
--output_model=path/to/save \
--quant_format=QDQ
```

## 2. Benchmark

```bash
bash run_benchmark.sh --input_model=path/to/model \ # model path as *.onnx
--dataset_location=/path/to/imagenet \
--label_path=/path/to/val.txt \
--mode=performance # or accuracy
```
Loading

0 comments on commit a4410a2

Please sign in to comment.