- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enhancement: Dynamic Context Token Size for OpenRouter LLM #1698
Comments
You can't, but OpenRouter does return context token size when fetching models. I've been thinking on how to use this and will have an update on this soon |
LibreChat is already doing what OpenRouter would do if you exceed the context though.
this is not really true. They just discard your tokens with the “middle-out” strategy if you go over. Disabling it would run you into context length error. LibreChat discards tokens that would exceed the context, starting from the tail end of the conversation |
What is your question?
Hi,
I'm testing codellama/codellama-70b-instruct LLM from OpenRouter. It's limited to a context length of 2,048 tokens. But it's capable for unlimited context length.
My questions are:
More Details
Link to LLM on OpenRouter: https://openrouter.ai/models/codellama/codellama-70b-instruct
Thank you for your help in advance,
Danny
What is the main subject of your question?
No response
Screenshots
Code of Conduct
The text was updated successfully, but these errors were encountered: