Implement BYOL

[prigoyal]

🌟 New SSL approach addition

Approach description

BYOL https://arxiv.org/abs/2006.07733

Open source status

• the model implementation is available: (give details)
• the model weights are available: (give details)
• who are the authors: (mention them, if possible by @gh-username)

[chaoyanghe]

Hi @prigoyal do you need help with the BYOL baseline

[prigoyal]

Hi @chaoyanghe , thank you so much for reaching out. Yes , absolutely would love help. Feel free to grab the issue and assign to yourself (or someone) :)

Highly appreciated and the contribution will be very impactful for the SSL community!!

cc @QuentinDuval

[chaoyanghe]

@prigoyal ok. I need time to understand your code first. Currently, I am working on BYOL with another idea, so it seems I can contribute the baseline to your framework once I understand your code well.

[prigoyal]

@chaoyanghe that sounds great. Just sharing some pointers that could be helpful:

In the VISSL docs, I created several flowcharts https://vissl.readthedocs.io/en/latest/flowcharts/train_workflow.html that could be helpful to understand the execution.

Further look at how to extend several components of VISSL in the docs https://vissl.readthedocs.io/en/latest/extend_modules/data_transforms.html

My understanding with BYOL is that : we would need to ensure that the data transforms are correct + make sure the loss is correct , the head of the model is correct -> all these components are modular and the docs above will be helpful.

Also look at the MoCo hook https://github.com/facebookresearch/vissl/blob/master/vissl/hooks/moco_hooks.py and moco loss https://github.com/facebookresearch/vissl/blob/master/vissl/losses/moco_loss.py for how we handled the momentum encoder

Lastly, feel free to reach out on this issue anytime :) thank you again! :)

[prigoyal]

cc @iseessel