Skip to content

Commit

Permalink
[Feature] Support FastFCN (open-mmlab#885)
Browse files Browse the repository at this point in the history
* FastFCN first commit

* FastFCN first commit

* Fixing lint error

* Fixing lint error

* use for loop on JPU

* Use For Loop

* Refactor FastFCN

* FastFCN

* FastFCN

* temp

* Uploading models & logs (4x4)

* Fixing typos

* fix typos

* rename config

* change README.md

* use _delete_=True

* change configs

* change start_level to 0

* change start_level to 0

* jpu

* add unittest for start_level!=0
  • Loading branch information
MengzhangLI authored Sep 30, 2021
1 parent 7db1cbb commit a9d1295
Show file tree
Hide file tree
Showing 15 changed files with 466 additions and 1 deletion.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ Supported methods:
- [x] [DMNet (ICCV'2019)](configs/dmnet)
- [x] [ANN (ICCV'2019)](configs/ann)
- [x] [GCNet (ICCVW'2019/TPAMI'2020)](configs/gcnet)
- [x] [FastFCN (ArXiv'2019)](configs/fastfcn)
- [x] [Fast-SCNN (ArXiv'2019)](configs/fastscnn)
- [x] [ISANet (ArXiv'2019/IJCV'2021)](configs/isanet)
- [x] [OCRNet (ECCV'2020)](configs/ocrnet)
Expand Down
1 change: 1 addition & 0 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ MMSegmentation 是一个基于 PyTorch 的语义分割开源工具箱。它是 O
- [x] [DMNet (ICCV'2019)](configs/dmnet)
- [x] [ANN (ICCV'2019)](configs/ann)
- [x] [GCNet (ICCVW'2019/TPAMI'2020)](configs/gcnet)
- [x] [FastFCN (ArXiv'2019)](configs/fastfcn)
- [x] [Fast-SCNN (ArXiv'2019)](configs/fastscnn)
- [x] [ISANet (ArXiv'2019/IJCV'2021)](configs/isanet)
- [x] [OCRNet (ECCV'2020)](configs/ocrnet)
Expand Down
53 changes: 53 additions & 0 deletions configs/_base_/models/fastfcn_r50-d32_jpu_psp.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# model settings
norm_cfg = dict(type='SyncBN', requires_grad=True)
model = dict(
type='EncoderDecoder',
pretrained='open-mmlab://resnet50_v1c',
backbone=dict(
type='ResNetV1c',
depth=50,
num_stages=4,
dilations=(1, 1, 2, 4),
strides=(1, 2, 2, 2),
out_indices=(1, 2, 3),
norm_cfg=norm_cfg,
norm_eval=False,
style='pytorch',
contract_dilation=True),
neck=dict(
type='JPU',
in_channels=(512, 1024, 2048),
mid_channels=512,
start_level=0,
end_level=-1,
dilations=(1, 2, 4, 8),
align_corners=False,
norm_cfg=norm_cfg),
decode_head=dict(
type='PSPHead',
in_channels=2048,
in_index=2,
channels=512,
pool_scales=(1, 2, 3, 6),
dropout_ratio=0.1,
num_classes=19,
norm_cfg=norm_cfg,
align_corners=False,
loss_decode=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
auxiliary_head=dict(
type='FCNHead',
in_channels=1024,
in_index=1,
channels=256,
num_convs=1,
concat_input=False,
dropout_ratio=0.1,
num_classes=19,
norm_cfg=norm_cfg,
align_corners=False,
loss_decode=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.4)),
# model training and testing settings
train_cfg=dict(),
test_cfg=dict(mode='whole'))
41 changes: 41 additions & 0 deletions configs/fastfcn/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation

## Introduction

<!-- [ALGORITHM] -->

<a href="https://github.com/wuhuikai/FastFCN">Official Repo</a>

<a href="https://github.com/open-mmlab/mmsegmentation/blob/v0.18.0/mmseg/models/necks/jpu.py#L12">Code Snippet</a>

<details>
<summary align="right"><a href="https://arxiv.org/abs/1903.11816">FastFCN (ArXiv'2019) </a></summary>

```latex
@article{wu2019fastfcn,
title={Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation},
author={Wu, Huikai and Zhang, Junge and Huang, Kaiqi and Liang, Kongming and Yu, Yizhou},
journal={arXiv preprint arXiv:1903.11816},
year={2019}
}
```

</details>

## Results and models

### Cityscapes

| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | mIoU | mIoU(ms+flip) | config | download |
| --------- | --------- | --------- | ------: | -------- | -------------- | ----: | ------------- | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| DeepLabV3 + JPU | R-50-D32 | 512x1024 | 80000 | 5.67 | 2.64 | 79.12 | 80.58 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes_20210928_053722-5d1a2648.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes_20210928_053722.log.json) |
| DeepLabV3 + JPU (4x4) | R-50-D32 | 512x1024 | 80000 | 9.79 | - | 79.52 | 80.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes_20210924_214357-72220849.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes_20210924_214357.log.json) |
| PSPNet + JPU | R-50-D32 | 512x1024 | 80000 | 5.67 | 4.40 | 79.26 | 80.86 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes_20210928_053722-57749bed.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes_20210928_053722.log.json) |
| PSPNet + JPU (4x4) | R-50-D32 | 512x1024 | 80000 | 9.94 | - | 78.76 | 80.03 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes_20210925_061841-77e87b0a.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes_20210925_061841.log.json) |
| EncNet + JPU | R-50-D32 | 512x1024 | 80000 | 8.15 | 4.77 | 77.97 |79.92 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes_20210928_030036-78da5046.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes_20210928_030036.log.json) |
| EncNet + JPU (4x4)| R-50-D32 | 512x1024 | 80000 | 15.45 | - | 78.6 | 80.25 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes_20210926_093217-e1eb6dbb.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes_20210926_093217.log.json) |

Note:

- `4x4` means 4 GPUs with 4 samples per GPU in training, default setting is 4 GPUs with 2 samples per GPU in training.
- Results of [DeepLabV3 (mIoU: 79.32)](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/deeplabv3), [PSPNet (mIoU: 78.55)](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/pspnet) and [ENCNet (mIoU: 77.94)](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/encnet) can be found in each original repository.
126 changes: 126 additions & 0 deletions configs/fastfcn/fastfcn.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
Collections:
- Name: fastfcn
Metadata:
Training Data:
- Cityscapes
Paper:
URL: https://arxiv.org/abs/1903.11816
Title: 'FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation'
README: configs/fastfcn/README.md
Code:
URL: https://github.com/open-mmlab/mmsegmentation/blob/v0.18.0/mmseg/models/necks/jpu.py#L12
Version: v0.18.0
Converted From:
Code: https://github.com/wuhuikai/FastFCN
Models:
- Name: fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes
In Collection: fastfcn
Metadata:
backbone: R-50-D32
crop size: (512,1024)
lr schd: 80000
inference time (ms/im):
- value: 378.79
hardware: V100
backend: PyTorch
batch size: 1
mode: FP32
resolution: (512,1024)
memory (GB): 5.67
Results:
- Task: Semantic Segmentation
Dataset: Cityscapes
Metrics:
mIoU: 79.12
mIoU(ms+flip): 80.58
Config: configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes_20210928_053722-5d1a2648.pth
- Name: fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes
In Collection: fastfcn
Metadata:
backbone: R-50-D32
crop size: (512,1024)
lr schd: 80000
memory (GB): 9.79
Results:
- Task: Semantic Segmentation
Dataset: Cityscapes
Metrics:
mIoU: 79.52
mIoU(ms+flip): 80.91
Config: configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes.py
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes_20210924_214357-72220849.pth
- Name: fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes
In Collection: fastfcn
Metadata:
backbone: R-50-D32
crop size: (512,1024)
lr schd: 80000
inference time (ms/im):
- value: 227.27
hardware: V100
backend: PyTorch
batch size: 1
mode: FP32
resolution: (512,1024)
memory (GB): 5.67
Results:
- Task: Semantic Segmentation
Dataset: Cityscapes
Metrics:
mIoU: 79.26
mIoU(ms+flip): 80.86
Config: configs/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes_20210928_053722-57749bed.pth
- Name: fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes
In Collection: fastfcn
Metadata:
backbone: R-50-D32
crop size: (512,1024)
lr schd: 80000
memory (GB): 9.94
Results:
- Task: Semantic Segmentation
Dataset: Cityscapes
Metrics:
mIoU: 78.76
mIoU(ms+flip): 80.03
Config: configs/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes.py
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes_20210925_061841-77e87b0a.pth
- Name: fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes
In Collection: fastfcn
Metadata:
backbone: R-50-D32
crop size: (512,1024)
lr schd: 80000
inference time (ms/im):
- value: 209.64
hardware: V100
backend: PyTorch
batch size: 1
mode: FP32
resolution: (512,1024)
memory (GB): 8.15
Results:
- Task: Semantic Segmentation
Dataset: Cityscapes
Metrics:
mIoU: 77.97
mIoU(ms+flip): 79.92
Config: configs/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes_20210928_030036-78da5046.pth
- Name: fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes
In Collection: fastfcn
Metadata:
backbone: R-50-D32
crop size: (512,1024)
lr schd: 80000
memory (GB): 15.45
Results:
- Task: Semantic Segmentation
Dataset: Cityscapes
Metrics:
mIoU: 78.6
mIoU(ms+flip): 80.25
Config: configs/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes.py
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes_20210926_093217-e1eb6dbb.pth
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# model settings
_base_ = './fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py'
data = dict(
samples_per_gpu=4,
workers_per_gpu=4,
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# model settings
_base_ = './fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py'
norm_cfg = dict(type='SyncBN', requires_grad=True)
model = dict(
decode_head=dict(
_delete_=True,
type='ASPPHead',
in_channels=2048,
in_index=2,
channels=512,
dilations=(1, 12, 24, 36),
dropout_ratio=0.1,
num_classes=19,
norm_cfg=norm_cfg,
align_corners=False,
loss_decode=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
# model training and testing settings
train_cfg=dict(),
test_cfg=dict(mode='whole'))
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# model settings
_base_ = './fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py'
data = dict(
samples_per_gpu=4,
workers_per_gpu=4,
)
24 changes: 24 additions & 0 deletions configs/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# model settings
_base_ = './fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py'
norm_cfg = dict(type='SyncBN', requires_grad=True)
model = dict(
decode_head=dict(
_delete_=True,
type='EncHead',
in_channels=[512, 1024, 2048],
in_index=(0, 1, 2),
channels=512,
num_codes=32,
use_se_loss=True,
add_lateral=False,
dropout_ratio=0.1,
num_classes=19,
norm_cfg=norm_cfg,
align_corners=False,
loss_decode=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_se_decode=dict(
type='CrossEntropyLoss', use_sigmoid=True, loss_weight=0.2)),
# model training and testing settings
train_cfg=dict(),
test_cfg=dict(mode='whole'))
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
_base_ = [
'../_base_/models/fastfcn_r50-d32_jpu_psp.py',
'../_base_/datasets/cityscapes.py', '../_base_/default_runtime.py',
'../_base_/schedules/schedule_80k.py'
]
data = dict(
samples_per_gpu=4,
workers_per_gpu=4,
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
_base_ = [
'../_base_/models/fastfcn_r50-d32_jpu_psp.py',
'../_base_/datasets/cityscapes.py', '../_base_/default_runtime.py',
'../_base_/schedules/schedule_80k.py'
]
3 changes: 2 additions & 1 deletion mmseg/models/necks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Copyright (c) OpenMMLab. All rights reserved.
from .fpn import FPN
from .ic_neck import ICNeck
from .jpu import JPU
from .mla_neck import MLANeck
from .multilevel_neck import MultiLevelNeck

__all__ = ['FPN', 'MultiLevelNeck', 'MLANeck', 'ICNeck']
__all__ = ['FPN', 'MultiLevelNeck', 'MLANeck', 'ICNeck', 'JPU']
Loading

0 comments on commit a9d1295

Please sign in to comment.