Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ai: update llm tokens used #3372

Merged
merged 16 commits into from
Jan 28, 2025
Merged

ai: update llm tokens used #3372

merged 16 commits into from
Jan 28, 2025

Conversation

rickstaa
Copy link
Member

@rickstaa rickstaa commented Jan 27, 2025

What does this pull request do? Explain your changes. (required)

Important

Merge #3336 and livepeer/ai-worker#402 first and release runner v0.13.2.

This pull request updates the codebase to use the new LLM OpenAPI spec, which replaces tokens_used with usage. See livepeer/ai-worker#402 for more details.

Specific updates (required)

  • Updates ai_process.go to use the new ai-worker client bindings.
  • Updates LLM tests

How did you test each of these updates (required)

Run an on-chain Gateway and Orchestrator.

Does this pull request close any open issues?

No, but it implements the change discussed in https://discordapp.com/channels/423160867534929930/1319248702032056383/1332619710562439168.

Checklist:

@github-actions github-actions bot added dependencies Pull requests that update a dependency file go Pull requests that update Go code AI Issues and PR related to the AI-video branch. labels Jan 27, 2025
@rickstaa rickstaa changed the title ai update llm tokens used ai: update llm tokens used Jan 27, 2025
Copy link
Collaborator

@ad-astra-video ad-astra-video left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with updated go.sum after merger livepeer/ai-worker PR #402 locally streaming and non-streaming responses.

Copy link

codecov bot commented Jan 28, 2025

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 33.69623%. Comparing base (1065885) to head (d849d51).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
server/ai_process.go 0.00000% 3 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master       #3372         +/-   ##
===================================================
+ Coverage   33.69179%   33.69623%   +0.00444%     
===================================================
  Files            141         141                 
  Lines          37475       37476          +1     
===================================================
+ Hits           12626       12628          +2     
+ Misses         24127       24126          -1     
  Partials         722         722                 
Files with missing lines Coverage Δ
server/ai_process.go 0.59880% <0.00000%> (-0.00052%) ⬇️

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1065885...d849d51. Read the comment docs.

Files with missing lines Coverage Δ
server/ai_process.go 0.59880% <0.00000%> (-0.00052%) ⬇️

... and 2 files with indirect coverage changes

@ad-astra-video ad-astra-video merged commit ede7d21 into master Jan 28, 2025
18 checks passed
@ad-astra-video ad-astra-video deleted the ai_update_llm_tokens_used branch January 28, 2025 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AI Issues and PR related to the AI-video branch. dependencies Pull requests that update a dependency file go Pull requests that update Go code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants