community: Support Anthropics Claude V3 Sonnet with Images and Text in Bedrock #18579

Adibuer-lab · 2024-03-05T14:56:46Z

PR Enhancements to Support Anthropics' Claude V3 Sonnet Model

This PR introduces enhancements to the Bedrock class within the community package to support Anthropics' Claude V3 Sonnet model, enabling the handling of both images and text in a single request.

Key Changes

Addition of _prepare_messages_api_input Function: Constructs the messages part of the Claude V3 request format, supporting text and optional base64-encoded images.
Addition of _convert_human_assistant_to_api_format Function: converts text completion api inputs in human-asssistant format to messages api input format, only supporting text
Modifications to LLMInputOutputAdapter.prepare_input: Conditionally formats requests for Claude V3, including specifying anthropic_version and handling max_tokens appropriately.
Adjustments to Response Processing: Accommodate variations in the response structure between Claude V2 and Claude V3.

Issue: N/A

Dependencies: No new dependencies were introduced.

Twitter Handle: N/A

Testing and Documentation

Included unit tests verify the correct construction of the Claude V3 messages format.
Also tested by running llm.invoke against bedrock API using AWS account to validate the new request and response handling logic within the Bedrock class.

Example

llm = Bedrock(
    credentials_profile_name="",
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    model_kwargs={"max_tokens": 4096},
)
 
test = llm.invoke(
    '{"messages":[{'
        '"role":"user","content":[{"type":"text","text":'
        '"Can you give me some travel tips for Japan?"}]},'
        '{"role":"assistant","content":[{"type":"text","text":'
        '"Absolutely! What do you need?"}]},'
        '{"role":"user","content":[{"type":"text","text":"Mostly '
        'interested in food and sightseeing."}]}'
        "]}"
    )

print(test)

Request Body

{
  "body": {
    "max_tokens": 4096,
    "anthropic_version": "bedrock-2023-05-31",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Can you give me some travel tips for Japan?"
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "Absolutely! What do you need?"
          }
        ]
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Mostly interested in food and sightseeing."
          }
        ]
      }
    ]
  },
  "modelId": "anthropic.claude-3-sonnet-20240229-v1:0",
  "accept": "application/json",
  "contentType": "application/json"
}

Extracted Text from Response

Here are some great travel tips for food and sightseeing in Japan:

Food:

Try traditional Japanese cuisines like sushi, ramen, yakitori (grilled skewers), okonomiyaki (savory pancakes), and regional specialties.
Eat at Japanese diners (shokudo) for affordable, delicious local fare.
Sample street foods like takoyaki (octopus balls), yakisoba (fried noodles), and taiyaki (fish-shaped cakes).
Don't miss out on fresh seafood if visiting coastal cities.
Look for long lines or crowded restaurants, which often indicate amazing food.

Sightseeing:

In Tokyo, see the Imperial Palace, Sensoji Temple, Shibuya Crossing, Tsukiji Fish Market, and alternative neighborhoods like Harajuku and Akihabara.
In Kyoto, explore Fushimi Inari Shrine, Kinkakuji (Golden Pavilion), Arashiyama bamboo forest, and Nijo Castle.
Visit iconic Mt. Fuji and consider hiking part of it during climbing season.
Experience a traditional ryokan (Japanese inn) with onsen (hot spring baths).
Take a day trip to places like Nara, Hiroshima, Miyajima Island, or Nikko.
For off-the-beaten-path destinations, consider Yakushima Island, Okinawa, Tottori Sand Dunes, or Shikoku Pilgrimage routes.

Some general tips are to get a JR Rail Pass if traveling long distances, try capsule hotels for budget stays, carry a pocket WiFi device, and learn a few basic Japanese phrases. Let me know if you need any other Japan tips!

If system prompt is to be included:

test = llm.invoke(
    '{"messages":[{'
    '"role":"user","content":[{"type":"text","text":'
    '"Can you give me some travel tips for Japan?"}]},'
    '{"role":"assistant","content":[{"type":"text","text":'
    '"Absolutely! What do you need?"}]},'
    '{"role":"user","content":[{"type":"text","text":"Mostly '
    'interested in food and sightseeing."}]}'
    "]}",
    kwargs={"system": "Reply only in Spanish"},
)

print(test)

Request body becomes:

{
    "body": {
        "max_tokens": 4096,
        "anthropic_version": "bedrock-2023-05-31",
        "system": "Reply only in Spanish",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Can you give me some travel tips for Japan?"
                    }
                ]
            },
             ...,
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Mostly interested in food and sightseeing."
                    }
                ]
            }
        ]
    },
    "modelId": "anthropic.claude-3-sonnet-20240229-v1:0",
    "accept": "application/json",
    "contentType": "application/json"
}

Response:
Claro, Japón es un destino increíble para la comida y los sitios turísticos. Aquí te doy algunos consejos:

Para la comida:

Prueba el sushi y sashimi frescos en los mercados de pescado como el Tsukiji en Tokio. Es una experiencia única.
No te pierdas los platos tradicionales como el ramen, tempura, yakitori (brochetas de pollo) y okonomiyaki (tortita salada).
Visita las calles de comida como Dotonbori en Osaka para probar delicias callejeras.
Experimenta una cena kaiseki, una sofisticada comida tradicional de varios platos.

Para hacer turismo:

Visita templos y santuarios emblemáticos como el santuario Fushimi Inari en Kioto con sus miles de toriis (puertas rojas).
Contempla el monte Fuji, el pico más alto de Japón.
Recorre los jardines y castillos históricos como el castillo de Himeji.
Sumérjete en la cultura pop y la iluminación de Tokio en barrios como Shibuya y Akihabara.
Prueba las aguas termales onsen, una relajante experiencia japonesa.
Viaja en tren bala (shinkansen) para moverte rápido entre ciudades.

¡Hay tantísimo por hacer! Disfruta de la combinación única de tradición y modernidad en Japón.

With images in multiple messages:

import json

conversation = {
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Hello, I'm looking for some travel advice."}
            ],
        },
        {
            "role": "assistant",
            "content": [
                {
                    "type": "text",
                    "text": "Sure, I can help with that. What's your destination?",
                }
            ],
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "I'm planning a trip to Japan. Specifically, I'm interested in historical sites in Kyoto. Here is a place I would like to go.",
                },
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": "/9j/4AAQSkZJRgABAQAAAQABAA...",
                    },
                },
            ],
        },
        {
            "role": "assistant",
            "content": [
                {
                    "type": "text",
                    "text": """The image depicts the famous Kinkaku-ji Temple, also known as the Golden Pavilion, in Kyoto, Japan. This Zen Buddhist temple is one of Kyoto's most iconic attractions and a must-visit for anyone interested in historical sites.
The temple is a three-story building covered in brilliant gold leaf, elegantly situated beside a tranquil pond surrounded by well-manicured gardens. The top two floors are constructed in the traditional Japanese architectural style, while the first floor is inspired by Buddhist architecture.
The serene landscape, with the temple's golden reflection shimmering in the pond's waters, lush greenery, carefully placed rocks, and the backdrop of forested hills, exemplifies the aesthetic ideals of Japanese garden design.
Kinkaku-ji dates back to the 14th century and has survived fires and wars over the centuries. Visiting this visually stunning and culturally significant temple is definitely recommended when exploring Kyoto's rich history and traditional architecture.""",
                }
            ],
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Thanks! That was really helpful. Here's a photo from my last trip, by the way. Can I get from the place in my last trip to the previous image I shared.",
                },
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": "/9j/4AAQSkZJRgAB..."
                        },
                },
            ],
        },
    ]
}

# Convert the dictionary into a JSON string
json_string = json.dumps(conversation, indent=4)

test = llm.invoke(json_string)

print(test)

Request body becomes:

{
  "body": {
    "max_tokens": 4096,
    "anthropic_version": "bedrock-2023-05-31",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Hello, I'm looking for some travel advice."
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "Sure, I can help with that. What's your destination?"
          }
        ]
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "I'm planning a trip to Japan. Specifically, I'm interested in historical sites in Kyoto. Here is a place I would like to go."
          },
          {
            "type": "image",
            "source": {
              "type": "base64",
              "media_type": "image/jpeg",
              "data": "/9j/4AAQSkZJRgABAQAAAQABAA..."
            }
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "The image depicts the famous Kinkaku-ji Temple, also known as the Golden Pavilion, in Kyoto, Japan. This Zen Buddhist temple is one of Kyoto's most iconic attractions and a must-visit for anyone interested in historical sites.
The temple is a three-story building covered in brilliant gold leaf, elegantly situated beside a tranquil pond surrounded by well-manicured gardens. The top two floors are constructed in the traditional Japanese architectural style, while the first floor is inspired by Buddhist architecture.
The serene landscape, with the temple's golden reflection shimmering in the pond's waters, lush greenery, carefully placed rocks, and the backdrop of forested hills, exemplifies the aesthetic ideals of Japanese garden design.
Kinkaku-ji dates back to the 14th century and has survived fires and wars over the centuries. Visiting this visually stunning and culturally significant temple is definitely recommended when exploring Kyoto's rich history and traditional architecture."
          }
        ]
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Thanks! That was really helpful. Here's a photo from my last trip, by the way. Can I get from the place in my last trip to the previous image I shared."
          },
          {
            "type": "image",
            "source": {
              "type": "base64",
              "media_type": "image/jpeg",
              "data": "/9j/4AAQSkZJRgAB..."
            }
          }
        ]
      }
    ]
  },
  "modelId": "anthropic.claude-3-sonnet-20240229-v1:0",
  "accept": "application/json",
  "contentType": "application/json"
}

Extracted text from Response:
The image you provided depicts the famous Bamboo Grove in Arashiyama, a district located in northwestern Kyoto. This iconic bamboo forest is one of Kyoto's top attractions and a must-visit for many travelers.

While the Bamboo Grove and the Kinkaku-ji Golden Pavilion are located in different parts of Kyoto, it is possible to visit both sites on the same day with some planning.

Here are a few suggestions on getting from Arashiyama to Kinkaku-ji:

Take a bus or taxi from Arashiyama to Kyoto Station, then transfer to a city bus or subway that goes to the Kinkaku-ji area.
If you prefer scenic routes, you can take the Randen streetcar from Arashiyama to Shijo-Omiya, then transfer to a city bus bound for Kinkaku-ji.
Consider renting a bicycle in Arashiyama and cycling along the Katsura River towards Kinkaku-ji, which takes around 1-1.5 hours of mostly flat riding.

The travel time between the two sites can range from 40 minutes to 1.5 hours, depending on your chosen transportation method and routing. I'd advise allocating enough time to fully experience both the mystical Bamboo Grove and the majestic Golden Pavilion without feeling rushed.

Using old text completions api input but no support for image data with this:

test = llm.invoke(
    "Human: Hello, I'm looking for some travel advice.Assistant: Sure, I can help with that. What's your destination?Human: I'm planning a trip to Japan. Specifically, I'm interested in historical sites in Kyoto.Assistant: Kyoto is a great choice! I recommend visiting Kinkaku-ji, the Golden Pavilion. It's a historic Zen Buddhist temple that is a must-see.Human: That sounds amazing. Do you have any recommendations for places to eat as well?Assistant: Definitely. For a traditional Kyoto dining experience, try kaiseki cuisine. Kikunoi is a renowned restaurant offering this type of dining.Human: Thanks! That was really helpful. I'll make sure to check out both Kinkaku-ji and Kikunoi. Here's a photo from my last trip, by the way",
    )
print(test)

Request body becomes:

{
  "body": {
    "max_tokens": 4096,
    "anthropic_version": "bedrock-2023-05-31",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Hello, I'm looking for some travel advice."
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "Sure, I can help with that. What's your destination?"
          }
        ]
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "I'm planning a trip to Japan. Specifically, I'm interested in historical sites in Kyoto."
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "Kyoto is a great choice! I recommend visiting Kinkaku-ji, the Golden Pavilion. It's a historic Zen Buddhist temple that is a must-see."
          }
        ]
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "That sounds amazing. Do you have any recommendations for places to eat as well?"
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "Definitely. For a traditional Kyoto dining experience, try kaiseki cuisine. Kikunoi is a renowned restaurant offering this type of dining."
          }
        ]
      },
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Thanks! That was really helpful. I'll make sure to check out both Kinkaku-ji and Kikunoi. Here's a photo from my last trip, by the way"
          }
        ]
      }
    ]
  },
  "modelId": "anthropic.claude-3-sonnet-20240229-v1:0",
  "accept": "application/json",
  "contentType": "application/json"
}

Extracted text from Response:
Unfortunately, you did not successfully upload an image for me to analyze. But I'd be happy to take a look if you try uploading it again. In the meantime, I can continue providing Kyoto travel recommendations if you'd like.

Some other top historical sites to consider are:

Kiyomizudera Temple - This wooden temple dating back to 1633 has an incredible wooden stage that juts out over the hillside.
Fushimi Inari Taisha - The iconic orange torii gates line a walking path through the forest at this important Shinto shrine.
Nijo Castle - This ornate 17th century castle has "nightingale" floors designed to sound like birds chirping to warn of intruders.
Arashiyama Bamboo Grove - A peaceful, photogenic walking path cutting through towering bamboo stalks.

Let me know if you need any other Kyoto tips as you continue planning your historical/cultural highlights!

vercel · 2024-03-05T14:56:50Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Mar 6, 2024 0:56am

Adibuer-lab · 2024-03-05T15:53:19Z

closed since needs to be backwards compatible to human assistant format with conversation being a list and each message in list corresponding to new format

{
  "modelId": "anthropic.claude-3-sonnet-20240229-v1:0",
  "contentType": "application/json",
  "accept": "application/json",
  "body": {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1000,
    "messages": [{
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "base64",
            "media_type": "image/jpeg",
            "data": "iVBORw..."
          }
        },
        {
          "type": "text",
          "text": "What's in this image?"
        }
      ]
    }]
  }
}

baskaryan · 2024-03-05T17:38:31Z

cc @3coins

Adibuer-lab · 2024-03-05T19:10:30Z

@efriis should translate text completions api inputs to Messages API now

3coins · 2024-03-05T20:08:31Z

@Adibuer-lab
Is passing an image in the kwargs the right approach. Seems like it should be part of the regular messages. Also, is a check for v3 required, as the messages API seems to be backwards compatible with other models.

Adibuer-lab · 2024-03-05T20:17:06Z

@Adibuer-lab Is passing an image in the kwargs the right approach. Seems like it should be part of the regular messages. Also, is a check for v3 required, as the messages API seems to be backwards compatible with other models.

might not be the right approach to be very honest. Especially if you want to add an image to a message earlier in the conversation.

Also true if messages api is backwards compatible with v2 models, can potentially deprecate text completions api and use only messages api for all claude versions with no need to check version.

With these considerations might be better to change everything from the prompt level that is invoked by llm. Though right now prompts are expected to be purely strings. So potentially a json string that is parsed to json form then used in body?

…ages

Adibuer-lab · 2024-03-05T21:36:20Z

@Adibuer-lab Is passing an image in the kwargs the right approach. Seems like it should be part of the regular messages. Also, is a check for v3 required, as the messages API seems to be backwards compatible with other models.

@3coins changed input mode to json string. Keeping old claude_v2 using test completions api since removing that might break people currently using that api

efriis

Same comment as #18548 which seems to be working on similar issue - Given claude v3 models only support the messages api in anthropic, would it make sense to add this compatibility to BedrockChat instead? https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html

Adibuer-lab · 2024-03-06T00:46:33Z

Same comment as #18548 which seems to be working on similar issue - Given claude v3 models only support the messages api in anthropic, would it make sense to add this compatibility to BedrockChat instead? https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html

@efriis wouldn't that be a separate issue with it's own change for BedrockChat? Bedrock needs to be able to work with claude 3 and so does BedrockChat. I wouldn't that do together here- I'd do BedrockChat in a separate change.

BedrockChat also seems to use convert_messages_to_prompt_anthropic function from anthropic.py in chat_models to handle api format, rather than using any method from BedrockBase in llms which is another reason I'd separate the changes.

@efriis would be great to hear your thoughts on this:

BedrockBase api handling and BedrockChat api handling look to be independent of each other since BedrockChat uses a ChatPromptAdapter which is actually using convert_messages_to_prompt_anthropic function from anthropic.py in chat_models. Any changes to add messages api handling to BedrockChat would best go via that ChatPromptAdapter or even better in my opinion in the convert_messages_to_prompt_anthropic function in anthropic.py.

A similar thing could be done in llms where BedrockBase uses api handling functions from anthropic.py in llms, but that would be a larger change that encompasses not just claude v3 support and could introduce breaking changes, and I would prefer that be in its own change as well

efriis · 2024-03-06T02:25:35Z

Consolidating discussion in #18548 . If I turn out to be wrong (very possible) about how folks are using the bedrock integrations, and it makes sense to somehow convert the claude 3 chat-only models into text completion connectors like this, we can revisit whether this or the other one is a better implementation of the LLM (not chat model) version and reopen!

Fixes #18513. ## Description This PR attempts to fix the support for Anthropic Claude v3 models in BedrockChat LLM. The changes here has updated the payload to use the `messages` format instead of the formatted text prompt for all models; `messages` API is backwards compatible with all models in Anthropic, so this should not break the experience for any models. ## Notes The PR in the current form does not support the v3 models for the non-chat Bedrock LLM. This means, that with these changes, users won't be able to able to use the v3 models with the Bedrock LLM. I can open a separate PR to tackle this use-case, the intent here was to get this out quickly, so users can start using and test the chat LLM. The Bedrock LLM classes have also grown complex with a lot of conditions to support various providers and models, and is ripe for a refactor to make future changes more palatable. This refactor is likely to take longer, and requires more thorough testing from the community. Credit to PRs [18579](#18579) and [18548](#18548) for some of the code here. --------- Co-authored-by: Erick Friis <[email protected]>

Fixes langchain-ai#18513. ## Description This PR attempts to fix the support for Anthropic Claude v3 models in BedrockChat LLM. The changes here has updated the payload to use the `messages` format instead of the formatted text prompt for all models; `messages` API is backwards compatible with all models in Anthropic, so this should not break the experience for any models. ## Notes The PR in the current form does not support the v3 models for the non-chat Bedrock LLM. This means, that with these changes, users won't be able to able to use the v3 models with the Bedrock LLM. I can open a separate PR to tackle this use-case, the intent here was to get this out quickly, so users can start using and test the chat LLM. The Bedrock LLM classes have also grown complex with a lot of conditions to support various providers and models, and is ripe for a refactor to make future changes more palatable. This refactor is likely to take longer, and requires more thorough testing from the community. Credit to PRs [18579](langchain-ai#18579) and [18548](langchain-ai#18548) for some of the code here. --------- Co-authored-by: Erick Friis <[email protected]>

Fixes #18513. ## Description This PR attempts to fix the support for Anthropic Claude v3 models in BedrockChat LLM. The changes here has updated the payload to use the `messages` format instead of the formatted text prompt for all models; `messages` API is backwards compatible with all models in Anthropic, so this should not break the experience for any models. ## Notes The PR in the current form does not support the v3 models for the non-chat Bedrock LLM. This means, that with these changes, users won't be able to able to use the v3 models with the Bedrock LLM. I can open a separate PR to tackle this use-case, the intent here was to get this out quickly, so users can start using and test the chat LLM. The Bedrock LLM classes have also grown complex with a lot of conditions to support various providers and models, and is ripe for a refactor to make future changes more palatable. This refactor is likely to take longer, and requires more thorough testing from the community. Credit to PRs [18579](langchain-ai/langchain#18579) and [18548](langchain-ai/langchain#18548) for some of the code here. --------- Co-authored-by: Erick Friis <[email protected]>

Adibuer-lab added 5 commits March 5, 2024 11:47

add claude3 support by preparing input appropriately

77fc841

remove image for now

92e82d4

got text working

d396cb6

got it working with images in kwargs

d57c745

get images working for claudv3

91b230f

dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. Ɑ: models Related to LLMs or chat model modules 🔌: anthropic Primarily related to Anthropic integrations 🤖:improvement Medium size change to existing code to handle new use-cases labels Mar 5, 2024

Adibuer-lab closed this Mar 5, 2024

Adibuer-lab reopened this Mar 5, 2024

messages dict

5ead9ce

Adibuer-lab closed this Mar 5, 2024

backwards compatibility to human assistant format

f430bc3

Adibuer-lab reopened this Mar 5, 2024

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Mar 5, 2024

Merge branch 'master' into master

c46f05a

Adibuer-lab mentioned this pull request Mar 5, 2024

community: Enable Claude v3 #18548

Closed

baskaryan assigned efriis Mar 5, 2024

baskaryan requested a review from efriis March 5, 2024 17:38

Adibuer-lab and others added 3 commits March 5, 2024 17:47

Merge branch 'langchain-ai:master' into master

bc95c1a

shorten lines for lint

7ca3467

add system prompt support

c8025f6

change input to json string to allow handling images in multiple mess…

7e5e5b9

…ages

Adibuer-lab and others added 4 commits March 5, 2024 21:57

maintain version chek for now while confirm backwards compatibility

5fff9b2

Merge branch 'master' into master

2c288c1

Merge branch 'master' into master

5436584

add convert text completions human-assistant input to messages api input

c064a91

efriis reviewed Mar 6, 2024

View reviewed changes

add return statement in case of unexpected inputs

0881b35

efriis closed this Mar 6, 2024

3coins mentioned this pull request Mar 6, 2024

Support for claude v3 models. #18630

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

community: Support Anthropics Claude V3 Sonnet with Images and Text in Bedrock #18579

community: Support Anthropics Claude V3 Sonnet with Images and Text in Bedrock #18579

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

vercel bot commented Mar 5, 2024 •

edited

Loading

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

baskaryan commented Mar 5, 2024

Adibuer-lab commented Mar 5, 2024

3coins commented Mar 5, 2024

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

efriis left a comment

Adibuer-lab commented Mar 6, 2024 •

edited

Loading

efriis commented Mar 6, 2024

community: Support Anthropics Claude V3 Sonnet with Images and Text in Bedrock #18579

community: Support Anthropics Claude V3 Sonnet with Images and Text in Bedrock #18579

Conversation

Adibuer-lab commented Mar 5, 2024 • edited Loading

PR Enhancements to Support Anthropics' Claude V3 Sonnet Model

Key Changes

Testing and Documentation

Example

Request Body

Extracted Text from Response

If system prompt is to be included:

With images in multiple messages:

Using old text completions api input but no support for image data with this:

vercel bot commented Mar 5, 2024 • edited Loading

Adibuer-lab commented Mar 5, 2024 • edited Loading

baskaryan commented Mar 5, 2024

Adibuer-lab commented Mar 5, 2024

3coins commented Mar 5, 2024

Adibuer-lab commented Mar 5, 2024 • edited Loading

Adibuer-lab commented Mar 5, 2024 • edited Loading

efriis left a comment

Choose a reason for hiding this comment

Adibuer-lab commented Mar 6, 2024 • edited Loading

efriis commented Mar 6, 2024

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

vercel bot commented Mar 5, 2024 •

edited

Loading

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

Adibuer-lab commented Mar 5, 2024 •

edited

Loading

Adibuer-lab commented Mar 6, 2024 •

edited

Loading