Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot reproduce the results on office31 and visda (vanilla) #10

Open
chester256 opened this issue Feb 15, 2020 · 2 comments
Open

Cannot reproduce the results on office31 and visda (vanilla) #10

chester256 opened this issue Feb 15, 2020 · 2 comments

Comments

@chester256
Copy link

Hi, the paper says you have used 10-crop during evaluation. But it didn't appear in the code. In addition, I cannot reproduce the results on both office31 and visda:

  1. For office31, I have tried 10-crop evalution and the result is 86.1. SAFN+ENT is 87.6 reported by the paper.
  2. For visda, I have tried both 10-crop and your default eval code (which is not 10-crop), and the results are 69.8 and 71.7 respectively (resnet101 base), which is far from the result reported by the paper.
    All the hyperparameters I have used are the defaults in the code. I have not tried imageCLEF. Hope you can help, thanks.
@chester256
Copy link
Author

Sorry that I found the base I have used is resnet50 in visda. I will update this issue if the problem still exist.

@jihanyang
Copy link
Owner

Hello, due to my recent plans, I don't have time and GPU to reproduce these experiments. However, you can send me an email and I would provide our 10-crop code for you. I hope that would be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants