Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models

Repository of the paper Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models, accepted at ML4AD@NeurIPS 2021.

Online Pipeline

The left side of the videos shows the ground truth data from CARLA. On the right you see the VAE based reconstructions. Videos are accelerated. For figure 6 in the paper the VAE model was trained with preprocessed lidar scans with a shape of 512x64 (same as for the images). This included some minor padding. After the submission we trained the VAE model with preprocessed lidar scans with a shape of 128x64 instead, which led to a much improved quality of the reconstructed pointclouds as you can see in the video.

online_pipeline.mp4

lidar_compression.mp4

Repository Structure

See the specific folders for additional information.

.
├── catkin_ws       # ROS workspace for running the online pipeline
├── evaluation      # Evaluation results
├── gan             # The GAN we use
├── lidar           # Contains the lidar preprocessing package and supplementary code
├── paper-graphics  # Code that generates some of our graphics
└── vae             # The VAE we use

Citation

If you find this code useful for your research, please cite our paper:

@article{Bogdoll_Compressing_2021_NeurIPS,
    author    = {Bogdoll, Daniel and Jestram, Johannes and Rauch, Jonas and Scheib, Christin and Wittig, Moritz and Z\"{o}llner, J. Marius},
    title     = {{Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models}},
    journal   = {NeurIPS Conference on Neural Information Processing Systems Workshop on Machine Learning for Autonomous Driving (ML4AD)},
    year      = {2021},
}