Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve llm benchmark page #637

Open
4 tasks
thibaultmol opened this issue Dec 21, 2024 · 1 comment
Open
4 tasks

Improve llm benchmark page #637

thibaultmol opened this issue Dec 21, 2024 · 1 comment
Labels
product:kagi_search Related to Kagi Search documentation status:needs_review This issue needs review by a Staff member.

Comments

@thibaultmol
Copy link
Contributor

Relevant Page(s)

https://help.kagi.com/kagi/ai/llm-benchmark.html

Description

  • Currently it's not possible to see which actual version of various llm's were tested. For example for gpt-4o . is it just that, is it latest, is it 2024-08-06. Same with the other models
  • It's not clear what the origin of the prices is for the various open source models. I assume it's Fireworks.ai pricing but it's not specified.
  • Related to the previous point, might be worth mentioning that those models can be hosted anyone so prices can vary
  • Pixtral isn't mentioned in the benchmarks but is available on The Assistant
@thibaultmol thibaultmol added product:kagi_search Related to Kagi Search documentation status:needs_review This issue needs review by a Staff member. labels Dec 21, 2024
@thibaultmol
Copy link
Contributor Author

cc @vprelovac probably

@thibaultmol thibaultmol mentioned this issue Jan 6, 2025
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
product:kagi_search Related to Kagi Search documentation status:needs_review This issue needs review by a Staff member.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@thibaultmol