Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update readme #221

Merged
merged 1 commit into from
Aug 29, 2023
Merged

update readme #221

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 53 additions & 87 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ FlagPerf是一款面向AI异构芯片的通用基准测试平台。我们希望
----------
### 支持列表

你可以点击**模型或训练框架**来跳转到**对应case的训练脚本**，✅来跳转到**对应厂商的运行配置**。

under review表示对应case的支持已开发完毕，在review中；Incoming表示正在添加或计划添加中；N/A表示不支持或尚无计划添加

#### 训练列表

你可以点击**模型或训练框架**来跳转到**对应case的训练脚本**，✅来跳转到**对应厂商的运行配置**。

<table width="960" border="0" cellpadding="0" cellspacing="0" style='width:960pt;border-collapse:collapse;table-layout:fixed;'>
<col width="73.60" style='mso-width-source:userset;mso-width-alt:3588;'/>
<col width="70" style='mso-width-source:userset;mso-width-alt:3413;'/>
Expand Down Expand Up @@ -43,7 +43,7 @@ under review表示对应case的支持已开发完毕，在review中；Incoming
<td class="xl69" x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/training/benchmarks/bert/paddle" style="text-decoration:none" target="_parent">Paddle</a></td>
<td class="xl69" x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/training/nvidia/bert-paddle" style="text-decoration:none" target="_parent">✅</a></td>
<td class="xl69" x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/training/kunlunxin/bert-paddle" style="text-decoration:none" target="_parent">✅</a></td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/training/iluvatar/bert-paddle" style="text-decoration:none" target="_parent">✅</a></td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
Expand Down Expand Up @@ -89,7 +89,7 @@ under review表示对应case的支持已开发完毕，在review中；Incoming
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str><a href="https://github.com/FlagOpen/FlagPerf/pull/144" style="text-decoration:none" target="_parent">under review</a></td>
<td class="xl69" x:str>under review</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="2" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/training/benchmarks/mobilenetv2" style="text-decoration:none" target="_parent">MobileNetV2</a></td>
Expand Down Expand Up @@ -227,7 +227,7 @@ under review表示对应case的支持已开发完毕，在review中；Incoming
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl69" x:str>Paddle</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>under review</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</a></td>
Expand Down Expand Up @@ -290,15 +290,15 @@ under review表示对应case的支持已开发完毕，在review中；Incoming
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>T5_small</td>
<td class="xl69" x:str>PyTorch</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>under review</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</a></td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>GPT2</td>
<td class="xl69" x:str>PyTorch</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>under review</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</a></td>
Expand Down Expand Up @@ -354,8 +354,11 @@ under review表示对应case的支持已开发完毕，在review中；Incoming
</table>



#### 推理列表

你可以点击**模型来跳转到**对应case的推理脚本及结果。

<table width="960" border="0" cellpadding="0" cellspacing="0" style='width:960pt;border-collapse:collapse;table-layout:fixed;'>
<col width="73.60" style='mso-width-source:userset;mso-width-alt:3588;'/>
<col width="70" style='mso-width-source:userset;mso-width-alt:3413;'/>
Expand All @@ -365,118 +368,81 @@ under review表示对应case的支持已开发完毕，在review中；Incoming
<tr height="16.80" class="xl65" style='height:16.80pt;'>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" x:str>模型</td>
<td class="xl65" x:str>精度</td>
<td class="xl65" x:str>英伟达tensorrt</td>
<td class="xl65" x:str>昆仑芯xtcl</td>
<td class="xl65" x:str>天数智芯ixrt</td>
<td class="xl65" x:str>模型</td>
<td class="xl65" x:str>英伟达+tensorrt</td>
<td class="xl65" x:str>英伟达+inductor</td>
<td class="xl65" x:str>昆仑芯+xtcl</td>
<td class="xl65" x:str>天数智芯+ixrt</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="2" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/resnet50" style="text-decoration:none" target="_parent">resnet50</a></td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="2" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/bertLarge" style="text-decoration:none" target="_parent">BertLarge</a></td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="2" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/vit_l_16" style="text-decoration:none" target="_parent">VisionTransformer</a></td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/resnet50" style="text-decoration:none" target="_parent">resnet50</a></td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>f32</td>
<td class="xl69" x:str>f16</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/bertLarge" style="text-decoration:none" target="_parent">BertLarge</a></td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>under review</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="2" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/yolov5" style="text-decoration:none" target="_parent">Yolov5_large</a></td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/vit_l_16" style="text-decoration:none" target="_parent">VisionTransformer</a></td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>under review</td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>✅</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/yolov5" style="text-decoration:none" target="_parent">Yolov5_large</a></td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="2" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/stable_diffusion_v1_4" style="text-decoration:none" target="_parent">Stable Diffusion v1.4</a></td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/stable_diffusion_v1_4" style="text-decoration:none" target="_parent">Stable Diffusion v1.4</a></td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>✅</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>SwinTransformer</td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/swinTransformer" style="text-decoration:none" target="_parent">SwinTransformer</td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>SAM(segment anything)</td>
<td class="xl69" x:str>float32</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>DLRM</td>
<td class="xl69" x:str>float32</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>DLRM</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>RNNT</td>
<td class="xl69" x:str>float32</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>RNNT</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>Llama2-7B</td>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str><a href="https://github.com/FlagOpen/FlagPerf/tree/main/inference/benchmarks/llama2_7b_mmlu" style="text-decoration:none" target="_parent">Llama2-7B-mmlu</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>f32/f16</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
</tr>
<tr height="16.80" style='height:16.80pt;'>
<td class="xl65" height="33.60" rowspan="1" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>Aquila-7B</td>
<td class="xl69" x:str>float16</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl65" height="33.60" style='height:33.60pt;border-right:none;border-bottom:none;' x:str>Aquila-7B-mmlu</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>Incoming</td>
<td class="xl69" x:str>N/A</td>
<td class="xl69" x:str>N/A</td>
</tr>
</table>

Expand Down