Skip to content

Latest commit

 

History

History
100 lines (84 loc) · 3.72 KB

README.md

File metadata and controls

100 lines (84 loc) · 3.72 KB

Wide & Deep

This document has instructions for how to run Wide & Deep benchmark for the following modes/precisions:

Benchmarking instructions and scripts for model training and inference other precisions are coming later.

FP32 Inference Instructions

  1. Clone tensorflow/models

    # We going to use older version of the tensorflow model repo.
$ git clone https://github.com/tensorflow/models.git
$ cd models
$ git checkout 6ff0a53f81439d807a78f8ba828deaea3aaaf269

  2. Download census income dataset from following link: https://archive.ics.uci.edu/ml/datasets/Census+Income

    Click on Download link and download adult.data and adult.test files

  3. Download and extract the pre-trained model.

$ wget https://storage.googleapis.com/intel-optimized-tensorflow/models/wide_deep_fp32_pretrained_model.tar.gz
$ tar -xzvf wide_deep_fp32_pretrained_model.tar.gz

  1. Clone the intelai/models repo. This repo has the launch script for running benchmarking, which we will use in the next step.

    $ git clone https://github.com/IntelAI/models.git

  2. How to run benchmarks

    • Running benchmarks in latency mode, set --batch-size = 1 
      $ cd /home/myuser/models/benchmarks

$ python launch_benchmark.py \ 
      --framework tensorflow \ 
      --model-source-dir /home/myuser/path/to/tensorflow-models \
      --precision fp32 \
      --mode inference \
      --model-name wide_deep \
      --batch-size 1 \
      --data-location /home/myuser/path/to/dataset \
      --checkpoint /home/myuser/path/to/wide_deep_fp32_pretrained_model \
      --docker-image intelaipg/intel-optimized-tensorflow:latest-devel-mkl \
      --verbose
    • Running benchmarks in throughput mode, set --batch-size = 1024 
      $ cd /home/myuser/models/benchmarks

$ python launch_benchmark.py \ 
      --framework tensorflow \ 
      --model-source-dir /home/myuser/path/to/tensorflow-models \
      --precision fp32 \
      --mode inference \
      --model-name wide_deep \
      --batch-size 1024 \
      --data-location /home/myuser/path/to/dataset \
      --checkpoint /home/myuser/path/to/wide_deep_fp32_pretrained_model \
      --docker-image intelaipg/intel-optimized-tensorflow:latest-devel-mkl \
      --verbose

  3. The log file is saved to the value of --output-dir.

    The tail of the log output when the benchmarking completes should look something like this:

    accuracy: 1.0
accuracy_baseline: 1.0
auc: 1.0
auc_precision_recall: 0.0
average_loss: 2.1470942e-05
global_step: 9775
label/mean: 0.0
loss: 2.1470942e-05
precision: 0.0
prediction/mean: 2.1461743e-05
recall: 0.0
End-to-End duration is %s 36.5971579552
Latency is: %s 0.00224784460139
lscpu_path_cmd = command -v lscpu
lscpu located here: /usr/bin/lscpu
current path: /workspace/benchmarks
search path: /workspace/benchmarks/*/tensorflow/wide_deep/inference/fp32/model_init.py
Using model init: /workspace/benchmarks/classification/tensorflow/wide_deep/inference/fp32/model_init.py
PYTHONPATH: :/workspace/models
RUNCMD: python common/tensorflow/run_tf_benchmark.py         --framework=tensorflow         --model-name=wide_deep         --precision=fp32         --mode=inference         --model-source-dir=/workspace/models         --intelai-models=/workspace/intelai_models         --batch-size=1                  --data-location=/dataset         --checkpoint=/checkpoints