-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add model ids corresponding to inference profiles supported by AWS Bedrock #5696
base: main
Are you sure you want to change the base?
Conversation
…drock AWS has something called "Inference Profiles" for performing cross-region inference for calling Bedrock APIs for specific models from specific regions. This is typically used by using particular model ids, which is added with this commit. Read more here: - https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html - https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html
…-profiles-supported-models Add model ids corresponding to inference profiles supported by AWS Bedrock
CodSpeed Performance ReportMerging #5696 will degrade performances by 32.42%Comparing Summary
Benchmarks breakdown
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if we added the field to pick a region insteaD?
@ogabrielluiz If you see in the linked docs, according to region + model, the model id should be changed. There doesn't seem to be a consistent logic there. It's just a table with values that need to be mapped. Hence I tried to keep it simple, since I'm blocked on this 😅 |
How should we proceed with this? Please advise |
AWS has something called "Inference Profiles" for performing cross-region inference for calling Bedrock APIs for specific models from specific regions. This is typically used by using particular model ids, which is added with this commit.
Read more here:
I noticed this when setting up Langflow and trying to use the models in Bedrock. It is a simple constants change for now. But maybe is it possible to allow a user to type-in a model id, if in the dropdown the option user tries to choose does not exist? Because API providers would keep adding new models every other week and we would need to keep playing catch up :)