[Feature] Support FastFCN

README.md

@@ -88,6 +88,7 @@ Supported methods:
 - [x] [DMNet (ICCV'2019)](configs/dmnet)
 - [x] [ANN (ICCV'2019)](configs/ann)
 - [x] [GCNet (ICCVW'2019/TPAMI'2020)](configs/gcnet)
+- [x] [FastFCN (ArXiv'2019)](configs/fastfcn)
 - [x] [Fast-SCNN (ArXiv'2019)](configs/fastscnn)
 - [x] [ISANet (ArXiv'2019/IJCV'2021)](configs/isanet)
 - [x] [OCRNet (ECCV'2020)](configs/ocrnet)

README_zh-CN.md

@@ -87,6 +87,7 @@ MMSegmentation 是一个基于 PyTorch 的语义分割开源工具箱。它是 O
 - [x] [DMNet (ICCV'2019)](configs/dmnet)
 - [x] [ANN (ICCV'2019)](configs/ann)
 - [x] [GCNet (ICCVW'2019/TPAMI'2020)](configs/gcnet)
+- [x] [FastFCN (ArXiv'2019)](configs/fastfcn)
 - [x] [Fast-SCNN (ArXiv'2019)](configs/fastscnn)
 - [x] [ISANet (ArXiv'2019/IJCV'2021)](configs/isanet)
 - [x] [OCRNet (ECCV'2020)](configs/ocrnet)

configs/_base_/models/fastfcn_r50-d32.py

@@ -0,0 +1,53 @@
+# model settings
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+model = dict(
+    type='EncoderDecoder',
+    pretrained='open-mmlab://resnet50_v1c',
+    backbone=dict(
+        type='ResNetV1c',
+        depth=50,
+        num_stages=4,
+        dilations=(1, 1, 2, 4),
+        strides=(1, 2, 2, 2),
+        out_indices=(1, 2, 3),
+        norm_cfg=norm_cfg,
+        norm_eval=False,
+        style='pytorch',
+        contract_dilation=True),
+    neck=dict(
+        type='JPU',
+        in_channels=(512, 1024, 2048),
+        mid_channels=512,
+        start_level=1,
+        end_level=-1,
+        dilations=(1, 2, 4, 8),
+        align_corners=False,
+        out_indices=(0, 1, 2),
+        norm_cfg=norm_cfg),
+    decode_head=dict(
+        type='PSPHead',
+        in_channels=2048,
+        in_index=2,
+        channels=512,
+        dropout_ratio=0.1,
+        num_classes=19,
+        norm_cfg=norm_cfg,
+        align_corners=False,
+        loss_decode=dict(
+            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
+    auxiliary_head=dict(
+        type='FCNHead',
+        in_channels=1024,
+        in_index=1,
+        channels=256,
+        num_convs=1,
+        concat_input=False,
+        dropout_ratio=0.1,
+        num_classes=19,
+        norm_cfg=norm_cfg,
+        align_corners=False,
+        loss_decode=dict(
+            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.4)),
+    # model training and testing settings
+    train_cfg=dict(),
+    test_cfg=dict(mode='whole'))

configs/fastfcn/README.md

@@ -0,0 +1,39 @@
+# FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation
+
+## Introduction
+
+<!-- [ALGORITHM] -->
+
+<a href="https://github.com/wuhuikai/FastFCN">Official Repo</a>
+
+<a href="https://github.com/open-mmlab/mmsegmentation/blob/v0.18.0/mmseg/models/necks/jpu.py#L12">Code Snippet</a>
+
+<details>
+<summary align="right"><a href="https://arxiv.org/abs/1903.11816">FastFCN (ArXiv'2019) </a></summary>
+
+```latex
+@article{wu2019fastfcn,
+title={Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation},
+author={Wu, Huikai and Zhang, Junge and Huang, Kaiqi and Liang, Kongming and Yu, Yizhou},
+journal={arXiv preprint arXiv:1903.11816},
+year={2019}
+}
+```
+
+</details>
+
+## Results and models
+
+### Cityscapes
+
+| Method    | Backbone  | Crop Size | Lr schd | Mem (GB) | Inf time (fps) |  mIoU | mIoU(ms+flip) | config                                                                                  | download                                                                                                                                                                                                                                                       |
+| --------- | --------- | --------- | ------: | -------- | -------------- | ----: | ------------- | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [DeepLabV3](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/deeplabv3) | R-50-D8         | 512x1024  |   80000 | 6.1        | 2.57              | 79.32 |         80.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/deeplabv3/deeplabv3_r50-d8_512x1024_80k_cityscapes.py)         | [model](https://download.openmmlab.com/mmsegmentation/v0.5/deeplabv3/deeplabv3_r50-d8_512x1024_80k_cityscapes/deeplabv3_r50-d8_512x1024_80k_cityscapes_20200606_113404-b92cfdd4.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/deeplabv3/deeplabv3_r50-d8_512x1024_80k_cityscapes/deeplabv3_r50-d8_512x1024_80k_cityscapes_20200606_113404.log.json) |
+| DeepLabV3 + JPU (4x2) | R-50-D32 | 512x1024  | 80000 | 999 | 999 | 0 | 0 | 0 |0 |
+| DeepLabV3 + JPU (4x4) | R-50-D32 | 512x1024  | 80000 | 9.79 | 2.64 | 79.52 | 80.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes_20210924_214357-c2b06737.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes_20210924_214357.log.json) |
+| [PSPNet](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/pspnet) | R-50-D8   | 512x1024  |   80000 | 6.1        | 4.07              | 78.55 |         79.79 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py)   | [model](https://download.openmmlab.com/mmsegmentation/v0.5/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes/pspnet_r50-d8_512x1024_80k_cityscapes_20200606_112131-2376f12b.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes/pspnet_r50-d8_512x1024_80k_cityscapes_20200606_112131.log.json)         |
+| PSPNet + JPU (4x2) | R-50-D32   | 512x1024  |   80000 |  5.67       |  4.40             | 79.26 |   80.86       |   |        |
+| PSPNet + JPU (4x2) | R-50-D32   | 512x1024  |   80000 |  9.94       |  -             | 78.76 |   80.03       | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn-jpu_psp_512x1024_80k_cityscapes.py)   | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_psp_512x1024_80k_cityscapes/fastfcn-jpu_psp_512x1024_80k_cityscapes_20210925_061841-b07c5e32.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_psp_512x1024_80k_cityscapes/fastfcn-jpu_psp_512x1024_80k_cityscapes_20210925_061841.log.json)         |
+| [encnet](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/encnet) | R-50-D8  | 512x1024  |   80000 | 8.6        | 4.58              | 77.94 |         79.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/encnet/encnet_r50-d8_512x1024_80k_cityscapes.py)  | [model](https://download.openmmlab.com/mmsegmentation/v0.5/encnet/encnet_r50-d8_512x1024_80k_cityscapes/encnet_r50-d8_512x1024_80k_cityscapes_20200622_003554-fc5c5624.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/encnet/encnet_r50-d8_512x1024_80k_cityscapes/encnet_r50-d8_512x1024_80k_cityscapes-20200622_003554.log.json)     |
+| encnet + JPU (4x2)| R-50-D32  | 512x1024  |   80000 | 8.15        |  4.77             | 77.97 |79.92          |   |    |
+| encnet + JPU (4x4)| R-50-D32  | 512x1024  |   80000 | 15.45        | -              | 78.6 |         80.25 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn-jpu_enc_512x1024_80k_cityscapes.py)  | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_enc_512x1024_80k_cityscapes/fastfcn-jpu_enc_512x1024_80k_cityscapes_20210926_093217-c2e5d0fd.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_enc_512x1024_80k_cityscapes/fastfcn-jpu_enc_512x1024_80k_cityscapes_20210926_093217.log.json)     |

configs/fastfcn/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes.py

@@ -0,0 +1,23 @@
+# model settings
+_base_ = './fastfcn-jpu_psp_512x1024_80k_cityscapes.py'
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+model = dict(
+    decode_head=dict(
+        type='ASPPHead',
+        in_channels=2048,
+        in_index=2,
+        channels=512,
+        dilations=(1, 12, 24, 36),
+        dropout_ratio=0.1,
+        num_classes=19,
+        norm_cfg=norm_cfg,
+        align_corners=False,
+        loss_decode=dict(
+            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
+    # model training and testing settings
+    train_cfg=dict(),
+    test_cfg=dict(mode='whole'))
+data = dict(
+    samples_per_gpu=4,
+    workers_per_gpu=4,
+)

configs/fastfcn/fastfcn-jpu_enc_512x1024_80k_cityscapes.py

@@ -0,0 +1,27 @@
+# model settings
+_base_ = './fastfcn-jpu_psp_512x1024_80k_cityscapes.py'
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+model = dict(
+    decode_head=dict(
+        type='EncHead',
+        in_channels=[512, 1024, 2048],
+        in_index=(0, 1, 2),
+        channels=512,
+        num_codes=32,
+        use_se_loss=True,
+        add_lateral=False,
+        dropout_ratio=0.1,
+        num_classes=19,
+        norm_cfg=norm_cfg,
+        align_corners=False,
+        loss_decode=dict(
+            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
+        loss_se_decode=dict(
+            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=0.2)),
+    # model training and testing settings
+    train_cfg=dict(),
+    test_cfg=dict(mode='whole'))
+data = dict(
+    samples_per_gpu=4,
+    workers_per_gpu=4,
+)

configs/fastfcn/fastfcn-jpu_psp_512x1024_80k_cityscapes.py

@@ -0,0 +1,8 @@
+_base_ = [
+    '../_base_/models/fastfcn_r50-d32.py', '../_base_/datasets/cityscapes.py',
+    '../_base_/default_runtime.py', '../_base_/schedules/schedule_80k.py'
+]
+data = dict(
+    samples_per_gpu=4,
+    workers_per_gpu=4,
+)

configs/fastfcn/fastfcn.yml

@@ -0,0 +1,199 @@
+Collections:
+- Name: fastfcn
+  Metadata:
+    Training Data:
+    - Cityscapes
+  Paper:
+    URL: https://arxiv.org/abs/1903.11816
+    Title: 'FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation'
+  README: configs/fastfcn/README.md
+  Code:
+    URL: https://github.com/open-mmlab/mmsegmentation/blob/v0.18.0/mmseg/models/necks/jpu.py#L12
+    Version: v0.18.0
+  Converted From:
+    Code: https://github.com/wuhuikai/FastFCN
+Models:
+- Name: ''
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D8
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 389.11
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 6.1
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 79.32
+      mIoU(ms+flip): 80.57
+  Config: ''
+  Weights: ''
+- Name: ''
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D32
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 1.0
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 999.0
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 0.0
+      mIoU(ms+flip): 0.0
+  Config: ''
+  Weights: ''
+- Name: fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D32
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 378.79
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 9.79
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 79.52
+      mIoU(ms+flip): 80.91
+  Config: configs/fastfcn/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes.py
+  Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes/fastfcn-jpu_deeplabv3_512x1024_80k_cityscapes_20210924_214357-c2b06737.pth
+- Name: ''
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D8
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 245.7
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 6.1
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 78.55
+      mIoU(ms+flip): 79.79
+  Config: ''
+  Weights: ''
+- Name: ''
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D32
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 227.27
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 5.67
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 79.26
+      mIoU(ms+flip): 80.86
+  Config: ''
+  Weights: ''
+- Name: fastfcn-jpu_psp_512x1024_80k_cityscapes
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D32
+    crop size: (512,1024)
+    lr schd: 80000
+    memory (GB): 9.94
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 78.76
+      mIoU(ms+flip): 80.03
+  Config: configs/fastfcn/fastfcn-jpu_psp_512x1024_80k_cityscapes.py
+  Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_psp_512x1024_80k_cityscapes/fastfcn-jpu_psp_512x1024_80k_cityscapes_20210925_061841-b07c5e32.pth
+- Name: ''
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D8
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 218.34
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 8.6
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 77.94
+      mIoU(ms+flip): 79.13
+  Config: ''
+  Weights: ''
+- Name: ''
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D32
+    crop size: (512,1024)
+    lr schd: 80000
+    inference time (ms/im):
+    - value: 209.64
+      hardware: V100
+      backend: PyTorch
+      batch size: 1
+      mode: FP32
+      resolution: (512,1024)
+    memory (GB): 8.15
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 77.97
+      mIoU(ms+flip): 79.92
+  Config: ''
+  Weights: ''
+- Name: fastfcn-jpu_enc_512x1024_80k_cityscapes
+  In Collection: fastfcn
+  Metadata:
+    backbone: R-50-D32
+    crop size: (512,1024)
+    lr schd: 80000
+    memory (GB): 15.45
+  Results:
+  - Task: Semantic Segmentation
+    Dataset: Cityscapes
+    Metrics:
+      mIoU: 78.6
+      mIoU(ms+flip): 80.25
+  Config: configs/fastfcn/fastfcn-jpu_enc_512x1024_80k_cityscapes.py
+  Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn-jpu_enc_512x1024_80k_cityscapes/fastfcn-jpu_enc_512x1024_80k_cityscapes_20210926_093217-c2e5d0fd.pth

mmseg/models/necks/__init__.py

@@ -1,6 +1,7 @@
 # Copyright (c) OpenMMLab. All rights reserved.
 from .fpn import FPN
+from .jpu import JPU
 from .mla_neck import MLANeck
 from .multilevel_neck import MultiLevelNeck
 
-__all__ = ['FPN', 'MultiLevelNeck', 'MLANeck']
+__all__ = ['FPN', 'MultiLevelNeck', 'MLANeck', 'JPU']