Skip to content

Latest commit

 

History

History
82 lines (72 loc) · 2.87 KB

README.md

File metadata and controls

82 lines (72 loc) · 2.87 KB

Sparse4D

Overall Architecture

[video demo](https://github.com/linxuewu/Sparse4D/releases/download/v0.0/video.avi)

nuScenes Benchmark

model dataset backbone img size mAP mATE mASE mAOE mAVE mAAE NDS ckpt log
Sparse4D validation Res101 640x1600 0.4409 0.6282 0.2721 0.3853 0.2922 0.1888 0.5438 ckpt log

Quick Start

Install requirements.

pip install -r requirements.txt

Download nuScenes dataset, pretrain checkpoint(fcos3d.pth ResNet101), pkl files(nuscenes_infos_trainval_with_inds.pkl) and init anchor centers(nuscenes_kmeans900.npy). Adjust the directory structure as follows:

Sparse4D
├── data
│   ├── nuscenes
│   │   ├── maps
│   │   ├── lidarseg
│   │   ├── samples
│   │   ├── sweeps
│   │   ├── v1.0-mini
│   │   ├── v1.0-test
|   |   └── v1.0-trainval
│   ├── nuscenes_cam
│   │   ├── nuscenes_infos_test.pkl
│   │   ├── nuscenes_infos_train.pkl
│   │   ├── nuscenes_infos_val.pkl
│   │   └── nuscenes_infos_trainval_with_inds.pkl
├── projects
│   ├── configs
│   │   ├── default_runtime.py
│   │   ├── sparse4d_r101_H1.py
│   │   ├── sparse4d_r101_H4.py
│   │   └── ...
│   └── mmdet3d_plugin
│       ├── apis
│       ├── core
│       ├── datasets
│       └── models
├── tools
│   ├── dist_test.sh
│   ├── dist_train.sh
│   ├── test.py
│   └── train.py
├── local_test.sh
├── local_train.sh
├── fcos3d.pth
└── nuscenes_kmeans900.npy

Train with config_name.py.

bash local_train.sh config_name

Test checkpoint_file with config_name.py.

bash local_test.sh config_name checkpoint_file

Citation

@misc{2211.10581,
  Author = {Xuewu Lin and Tianwei Lin and Zixiang Pei and Lichao Huang and Zhizhong Su},
  Title = {Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusion},
  Year = {2022},
  Eprint = {arXiv:2211.10581},
}

Acknowledgement