Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[iluvatar] swin_transformer-pytorch 1x1 2x8 #340

Merged
merged 8 commits into from
Dec 6, 2023
Merged

[iluvatar] swin_transformer-pytorch 1x1 2x8 #340

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b85b320
update iluvatar/swin_transformer-pytorch
Nov 28, 2023
1a880cf
update
Nov 29, 2023
9ed3e15
update
Nov 29, 2023
57cb65c
update
Nov 29, 2023
568c524
fix batch size mistake in readme
Nov 30, 2023
afa18e8
correct val_loss to final acc1
Nov 30, 2023
078b3d8
add finnal_acc1 and mem in readme
clveryang Dec 5, 2023
41d02ca
correct readme mem
clveryang Dec 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 29 additions & 7 deletions training/iluvatar/swin_transformer-pytorch/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,37 @@

- ##### 软件环境
- OS版本：Ubuntu 20.04
- OS kernel版本: 4.15.0-156-generic x86_64
- 加速卡驱动版本：3.0.0
- Docker 版本：20.10.8
- 训练框架版本：torch-1.10.2+corex.3.0.0
- OS kernel版本: 5.4.0-148-generic
- 加速卡驱动版本：3.1.0
- Docker 版本：24.0.5
- 训练框架版本：torch-1.13.1+corex.3.1.0
- 依赖软件版本：无


### 运行情况
| 训练资源 | 配置文件 | 运行时长(s) | 目标精度 | 收敛精度 | Steps数 | 性能（samples/s) |
| -------- | --------------- | ----------- | -------- | -------- | ------- | ---------------- |
| 单机8卡 | config_A100x1x8 | | | | | |
* 通用指标

| 指标名称 | 指标值 | 特殊说明 |
| -------------- | --------------------------------------------- | ------------------------------------------- |
| 任务类别 | Image Classification && Semantic Segmantation | |
| 模型 | swin_transformer | |
| 数据集 | Imagenet2012 1K | |
| 数据精度 | precision,见“性能指标” | 可选fp32/amp/fp16/tf32 |
| 超参修改 | fix_hp,见“性能指标” | 跑满硬件设备评测吞吐量所需特殊超参 |
| 硬件设备简称 | Iluvatar BI-V100 | |
| 硬件存储使用 | mem,见“性能指标” | 通常称为“显存”,单位为GiB |
| 端到端时间 | e2e_time,见“性能指标” | 总时间+Perf初始化等时间 |
| 总吞吐量 | p_whole,见“性能指标” | 实际训练样本数除以总时间(performance_whole) |
| 训练吞吐量 | p_train,见“性能指标” | 不包含每个epoch末尾的评估部分耗时 |
| **计算吞吐量** | **p_core,见“性能指标”** | 不包含数据IO部分的耗时(p3>p2>p1) |
| 训练结果 | final_acc1,见“性能指标” | 验证准确率 |
| 额外修改项 | 无 | |

* 性能指标

| 配置 | precision| fix_hp | e2e_time | p_whole | p_train | p_core | final_acc1 | mem |
|----------------------| ---------| ------ | ---------| ------- | ------- | ------ | -------- | --------- |
| BI-V100单机8卡(1x8) | amp | bs=352 | / | / | / | / | 81.23 | 26.8/32.0 |
| BI-V100单机单卡(1x1) | amp | bs=352 | / | / | / | / | / | 26.5/32.0 |
| BI-V100两机8卡(2x8) | amp | bs=352 | / | / | / | / | / | 26.8/32.0 |

4 changes: 4 additions & 0 deletions training/iluvatar/swin_transformer-pytorch/config/config_BI-V100x1x1.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from config_common import *

cfg = "configs/swin/swin_tiny_patch4_window7_224.yaml"
train_batch_size = 352
2 changes: 1 addition & 1 deletion training/iluvatar/swin_transformer-pytorch/config/config_BI-V100x1x8.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from config_common import *

cfg = "configs/swin/swin_tiny_patch4_window7_224.yaml"
train_batch_size = 256
train_batch_size = 352
4 changes: 4 additions & 0 deletions training/iluvatar/swin_transformer-pytorch/config/config_BI-V100x2x8.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from config_common import *

cfg = "configs/swin/swin_tiny_patch4_window7_224.yaml"
train_batch_size = 352
2 changes: 1 addition & 1 deletion training/nvidia/swin_transformer-pytorch/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
| 总吞吐量 | p_whole,见“性能指标” | 实际训练样本数除以总时间(performance_whole) |
| 训练吞吐量 | p_train,见“性能指标” | 不包含每个epoch末尾的评估部分耗时 |
| **计算吞吐量** | **p_core,见“性能指标”** | 不包含数据IO部分的耗时(p3>p2>p1) |
| 训练结果 | val_loss,见“性能指标” | 验证loss |
| 训练结果 | final_acc1,见“性能指标” | 验证准确率 |
| 额外修改项 | 无 | |

* 性能指标
Expand Down