Master Project

Improve Point Transformer by Bayesian Perturbation for Uncertainty Quantification

Uncertainty Quantification

• Reference: What Uncertainty Do we need in Computer Vision?

- Aleatoric(Data) vs. Epistemic(Model)

Data

• ModelNet40
  • Shape Classification
• S3DIS
  • Indoor Semantic Segmentation
• ShapeNet
  • Part Segmentation
• how to download?
  • see scripts/download.sh

Data Augmentation

Model Structure

Methods

Rank-one Bayesian Perturbation

Loss Function

File Structure

Note: The framework, taking reference in mmDectection, is a little bit clumsy and complicated. The basic idea is decouple different modules, and

├── config/                                                 
├── data/               
├── scripts/                                # Launcher
│   ├── ...
│   └── train.sh 
├── tools/                                  # Read Config
│   ├── ...
│   └── test.py
└── pointbnn/                               # Main Modules
    ├── engines/                            # Trainer, Tester, Hook
    ├── datasets/                           
    ├── model/                              
    └── utils/                              # misc

Training Details

• connect to gpu: srun --gres=gpu:2 --cpus-per-task=8 --pty --mail-type=ALL bash
• 2 RTX 2080 Ti

Experience records:

S3DIS: exp0: ptv3(vanilla), ce, lovasz, rpe, patch_size=64, crop_n_points=102400 exp3: bnn, bce, lovasz, rpe, patch_size=64, sto_type=['heads', 'proj'], crop_n_points=102400 exp5: bnn, bce, lovasz, no rpe, patch_size=128, sto_type=['heads', 'proj'], crop_n_points=102400

ModelNet40: exp0: ptv3(vanilla) exp7: bnn, no rpe, patch_size=128

Misc

• GEMM(General Matrix Multiplication)

  • A nice blog
  • core idea: partition

• A nice pre on Point Cloud processing

• Learn more about sparse convolution: Minkov Engine

• A nice paper to read: superpoint graph clustering

• How to infuse 2D feature to 3D, and how does it help? 2D fusion

• PointNet BNN

• Some people work on bayesian deep learning

• A paper repository for Point Cloud Understanding

The project structure is based on PointCept, Torch-Uncertainty