Can you just summarize a bit the decoder part?

[jizongFox]

@krishnabits001
Hello, Thanks for this wonderful work. I have checked your code for pretrain the decoder.
it was more than 800 lines within the same instance method, which is very hard to understand. Is it possible to have some words to summarize it a bit?
In the paper, the descriptions are not clear too.

[krishnabits001]

Hi,
Thank you for your interest.
Currently, we have few comments in both the models.py and pretr_decoder_local_contrastive_loss.py files to describe the intuition and corresponding implementation for pre-training the decoder.
Can you please let me know which exact line numbers in which files are not clear to you so that I can add more comments?

Also, we have the oral video presentation available now.
Can you please check from time-stamp 5.13 for the intuition behind pre-training the decoder?
https://neurips.cc/virtual/2020/protected/poster_949686ecef4ee20a62d16b4a2d7ccca3.html