[BFCL] Is this the same data used for the leaderboard?

[hexists]

hello.
Thank you for making the BFCL public.
I have a question. Is the data in “Berkeley-Feature-Currency-Leaderboard/Data” the same data you used for the leaderboard (https://gorilla.cs.berkeley.edu/leaderboard.html)?
I'm curious if the data used for the leaderboard is blind data, different from github.

Thank you in advance for your response.

[HuanzhiMao]

Yes, they are the same. There is no blind/private dataset used for the current version of the leaderboard.

[hexists]

@HuanzhiMao
Thank you for your response. I have one more question.
I was using a temperature of 0.7, but it seems like it would be better to go lower in terms of reproducibility, is there a reason you set it that way?
If you're not the person in charge, please don't bother answering.

[HuanzhiMao]

We are also considering setting the temperature to a lower value for reproducibility benefits. We would love to hear your opinions in #500!

[HuanzhiMao]

The temperature issue is addressed in #574, which was merged a long time ago. I forget to update here.