[ECCV'24] Online Vectorized HD Map Construction Using Geometry

Zhixin Zhang¹, Yiyuan Zhang², Xiaohan Ding³, Fusheng Jin^1*, Xiangyu Yue²

¹Beijing Institute of Technology,   ²CUHK,   ³Tencent AI Lab

Website | arXiv | YouTube | Bilibili | Zhihu

News

We're working on more powerful and efficient models, please stay tuned.

• (2024/7/2) GeMap is accepted by ECCV 2024 and we release a new GeMap model with 76.0 mAP.
• (2023/12/7) We released the first version of GeMap (with pre-trained checkpoints and evaluation).
• (2023/12/7) GeMap is released on arXiv.

Motivation

• Recent efforts have built strong baselines for online vectorized HD map construction task, however, shapes and relations of instances in urban road systems are still under-explored, such as parallelism, perpendicular, or rectangle-shape.

• As the ego vehicle moves, the shape of a specific instance or the relations between two instances will remain unchanged. To accurately represent such geometric features, invariance to rigid transformation is a fundamental property.

Highlights

This work contributes from two perspectives:

• GeMap achieves new state-of-the-art performance on the NuScenes and Argoverse 2 datasets. Remarkably, it reaches a 71.8% mAP on the large-scale Argoverse 2 dataset, outperforming MapTR V2 by +4.4% and surpassing the 70% mAP threshold for the first time.
• GeMap end-to-end learns Euclidean shapes and relations of map instances beyond basic perception. Specifically, we design a geometric loss based on angle and distance clues, which is robust to rigid transformations. We also decouple self-attention to independently handle Euclidean shapes and relations.

Quantitative Results

NuScenes

Model	Objective	Backbone	Epoch	mAP	FPS	Config / Log	Checkpoint
GeMap	simple	R50	110	62.7	15.6	config/log	model
GeMap	simple	Camera(R50) & LiDAR(SEC)	110	66.5	6.8	config/log	model
GeMap	full	R50	110	69.4	13.3	config/log	model
GeMap	full	Swin-T	110	72.0	10.0	config/log	model
GeMap	full	V2-99	110	72.2	9.5	config/log	model
GeMap	full	V2-99(DD3D)	110	76.0	9.5	config/log	model

Argoverse 2

Model	Objective	Backbone	Epoch	mAP	FPS	Config / Log	Checkpoint
GeMap	simple	R50	6	63.9	13.5	config/log	model
GeMap	simple	R50	24	68.2	13.5	config/log	model
GeMap	full	R50	24	71.8	12.1	config/log	model

* All models are trained on 8 NVIDIA RTX3090 GPUs. The speed (Frames Per Second, FPS) is evaluated on a single 3090 GPU.

Visualization Results

Comparison Video

GeMap exhibits more robust predictions in occluded and rotated scenarios, especially under rainy weather conditions.

demo_video_720p.mp4

More Cases of GeMap

Getting Started

• Installation
• Prepare Dataset
• Train and Eval
• Visualization

TODO

• Faster implementation for inference of GeMap.
• More powerful LiDAR and Camera + LiDAR models.
• Lighter and faster models with 30+ FPS.

Acknowledgements

GeMap is based on mmdetection3d. It is also greatly inspired by the following outstanding contributions to the open-source community: LSS, GKT, Swin-Transformer, VoVNet, BEVFormer, MapTR, BeMapNet, HDMapNet.

Citation

If the paper and code help your research, please kindly cite:

@article{zhang2023online,
  title={Online Vectorized HD Map Construction using Geometry},
  author={Zhang, Zhixin and Zhang, Yiyuan and Ding, Xiaohan and Jin, Fusheng and Yue, Xiangyu},
  journal={arXiv preprint arXiv:2312.03341},
  year={2023}
}