Skip to content
/ AMD Public

[CVPR 2024] Asymmetric Masked Distillation for Pre-Training Small Foundation Models

Notifications You must be signed in to change notification settings

MCG-NJU/AMD

Folders and files

NameName
Last commit message
Last commit date

Latest commit

author
JerryFlymi
Jun 4, 2024
8d2ec6f Β· Jun 4, 2024

History

3 Commits
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Jun 4, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024
Mar 30, 2024

Repository files navigation

[CVPR 2024] Official Implementation of AMD

flowchart The following ranking results are produced by vit-b:

PWC PWC PWC PWC PWC PWC

Asymmetric Masked Distillation for Pre-Training Small Foundation Models
Zhiyu Zhao, Bingkun Huang, Sen Xing, Gangshan Wu, Yu Qiao, and Limin Wang
Nanjing University, Shanghai AI Lab

News πŸ“°

[2024.3.27] Code and models have been released!
[2024.2.29] Code and models will be released in the following days.
[2024.2.27] AMD is accpeted by CVPR2024! πŸŽ‰πŸŽ‰πŸŽ‰

Main Results πŸš€

✨ Something-Something V2

Method Extra Data Backbone Resolution #Frames x Clips x Crops Top-1 Top-5
AMD no ViT-S 224x224 16x2x3 70.2 92.5
AMD no ViT-B 224x224 16x2x3 73.3 94.0

✨ Kinetics-400

Method Extra Data Backbone Resolution #Frames x Clips x Crops Top-1 Top-5
AMD no ViT-S 224x224 16x5x3 80.1 94.5
AMD no ViT-B 224x224 16x5x3 82.2 95.3

✨ AVA 2.2

Method Extra Data Extra Label Backbone #Frame x Sample Rate mAP
AMD Kinetics-400 βœ— ViT-B 16x4 29.9
AMD Kinetics-400 βœ“ ViT-B 16x4 33.5

✨ UCF101 & HMDB51

Method Extra Data Backbone UCF101 HMDB51
AMD Kinetics-400 ViT-B 97.1 79.6

✨ ImageNet-1K

Method Extra Data Backbone Resolution Top-1
AMD no ViT-S 224x224 82.1
AMD no ViT-B 224x224 84.6

Installation πŸ”¨

Please follow the instructions in INSTALL.md.

Data Preparation ➑️

Please follow the instructions in DATASET.md for data preparation.

Pre-training πŸ”„

The pre-training instruction is in PRETRAIN.md.

Fine-tuning ‴️

The fine-tuning instruction is in FINETUNE.md.

Model Zoo πŸ“

We provide pre-trained and fine-tuned models in MODEL_ZOO.md.

Acknowledgements πŸ‘

This project is built upon VideoMAEv2 and MGMAE. Thanks to the contributors of these great codebases.

Citation ✏️

If you find this repository useful, please use the following BibTeX entry for citation.

@misc{zhao2023amd,
      title={Asymmetric Masked Distillation for Pre-Training Small Foundation Models}, 
      author={Zhiyu Zhao and Bingkun Huang and Sen Xing and Gangshan Wu and Yu Qiao and Limin Wang},
      year={2023},
      eprint={2311.03149},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}