Possible data leak in lvis_v1_train_cat_info.json

[wusize]

Hi, Xinyi!

I loaded the file "lvis_v1_train_cat_info.json", it seems to contain image_count for rare classes. It may lead to data leak in the open-vocabulary setting when using the fed loss.

[xingyizhou]

Hi Size,

Thank you for bring up this. I remembered I noted this issue, but thought we could ignore this because the zero-shot embeddings in the FedLoss will never receive a positive loss. They will receive negative losses (and thus have a negative impact on the performance), but these are rare due to the design of Fedloss. I should have tried to remove them in the loss but it was likely not better since I used this in the final version. Please feel free to try the corrected version and post numbers if you find a considerable difference. Thanks!

Best,
Xingyi

[wusize]

OK. Thanks for your response! It seems not likely to influence the FedLoss. But I still recommend to push a correct version of lvis_v1_train_cat_info.json that records 0 for novel classes. Otherwise, when setting ignore_zero_class = True for ce and bce loss, there would be some problem.