Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Leaderboard: Make benchmark not beta and give them better descriptions #1838

Closed
x-tabdeveloping opened this issue Jan 20, 2025 · 1 comment
Closed

Leaderboard: Make benchmark not beta and give them better descriptions #1838

x-tabdeveloping opened this issue Jan 20, 2025 · 1 comment
Assignees
@x-tabdeveloping
Labels
leaderboard issues related to the leaderboard

Comments

@x-tabdeveloping
Copy link
Collaborator

Currently, many of the benchmarks in the new leaderboard have a beta label beside them. We should probably change this, and in the descriptions explain it clearer what the benchmarks are for.
For instance I can imagine that a new user would be confused as to what the difference is between MTEB(eng, classic) and MTEB(eng, beta) or why we made a new English benchmark in the first place.
@x-tabdeveloping x-tabdeveloping added the leaderboard issues related to the leaderboard label Jan 20, 2025
@x-tabdeveloping x-tabdeveloping self-assigned this Jan 21, 2025
@x-tabdeveloping x-tabdeveloping mentioned this issue Jan 21, 2025
Merged
@x-tabdeveloping
Copy link
Collaborator Author

Fixed in #1849

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@x-tabdeveloping x-tabdeveloping

Labels
leaderboard issues related to the leaderboard
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@x-tabdeveloping