Skip to content

Commit

Permalink
feat(client-bedrock-agent-runtime): Now supports streaming for inline…
Browse files Browse the repository at this point in the history 
… agents.
  • Loading branch information
awstools committed Jan 15, 2025
1 parent a550721 commit b6204f8
Show file tree
Hide file tree
Showing 5 changed files with 28 additions and 7 deletions.
6 changes: 3 additions & 3 deletions clients/client-bedrock-agent-runtime/src/commands/InvokeAgentCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ export interface InvokeAgentCommandOutput extends InvokeAgentResponse, __Metadat

/**
* <note>
* <p>The CLI doesn't support streaming operations in Amazon Bedrock, including <code>InvokeAgent</code>.</p>
* </note>
* </note>
* <p>Sends a prompt for the agent to process and respond to. Note the following fields for the request:</p>
* <ul>
* <li>
Expand Down Expand Up @@ -74,7 +73,8 @@ export interface InvokeAgentCommandOutput extends InvokeAgentResponse, __Metadat
* <p>In the <code>sessionState</code> object, you can include attributes for the session or prompt or, if you configured an action group to return control, results from invocation of the action group.</p>
* </li>
* </ul>
* <p>The response is returned in the <code>bytes</code> field of the <code>chunk</code> object.</p>
* <p>The response contains both <b>chunk</b> and <b>trace</b> attributes.</p>
* <p>The final response is returned in the <code>bytes</code> field of the <code>chunk</code> object. The <code>InvokeAgent</code> returns one chunk for the entire interaction.</p>
* <ul>
* <li>
* <p>The <code>attribution</code> object contains citations for parts of the response.</p>
Expand Down
7 changes: 5 additions & 2 deletions clients/client-bedrock-agent-runtime/src/commands/InvokeInlineAgentCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,7 @@ export interface InvokeInlineAgentCommandOutput extends InvokeInlineAgentRespons
* </li>
* </ul>
* <note>
* <p>The CLI doesn't support streaming operations in Amazon Bedrock, including <code>InvokeInlineAgent</code>.</p>
* </note>
* </note>
* @example
* Use a bare-bones client and the command you need to make an API call.
* ```javascript
Expand Down Expand Up @@ -298,6 +297,10 @@ export interface InvokeInlineAgentCommandOutput extends InvokeInlineAgentRespons
* latency: "standard" || "optimized",
* },
* },
* streamingConfigurations: { // StreamingConfigurations
* streamFinalResponse: true || false,
* applyGuardrailInterval: Number("int"),
* },
* };
* const command = new InvokeInlineAgentCommand(input);
* const response = await client.send(command);
Expand Down
11 changes: 11 additions & 0 deletions clients/client-bedrock-agent-runtime/src/models/models_0.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8650,6 +8650,17 @@ export interface InvokeInlineAgentRequest {
* @public
*/
bedrockModelConfigurations?: InlineBedrockModelConfigurations | undefined;

/**
* <p>
* Specifies the configurations for streaming.
* </p>
* <note>
* <p>To use agent streaming, you need permissions to perform the <code>bedrock:InvokeModelWithResponseStream</code> action.</p>
* </note>
* @public
*/
streamingConfigurations?: StreamingConfigurations | undefined;
}

/**
Expand Down
1 change: 1 addition & 0 deletions clients/client-bedrock-agent-runtime/src/protocols/Aws_restJson1.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,7 @@ export const se_InvokeInlineAgentCommand = async (
instruction: [],
knowledgeBases: (_) => se_KnowledgeBases(_, context),
promptOverrideConfiguration: (_) => se_PromptOverrideConfiguration(_, context),
streamingConfigurations: (_) => _json(_),
})
);
b.m("POST").h(headers).b(body);
Expand Down
10 changes: 8 additions & 2 deletions codegen/sdk-codegen/aws-models/bedrock-agent-runtime.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4657,7 +4657,7 @@
}
],
"traits": {
"smithy.api#documentation": "<note>\n <p>The CLI doesn't support streaming operations in Amazon Bedrock, including <code>InvokeAgent</code>.</p>\n </note>\n <p>Sends a prompt for the agent to process and respond to. Note the following fields for the request:</p>\n <ul>\n <li>\n <p>To continue the same conversation with an agent, use the same <code>sessionId</code> value in the request.</p>\n </li>\n <li>\n <p>To activate trace enablement, turn <code>enableTrace</code> to <code>true</code>. Trace enablement helps you follow the agent's reasoning process that led it to the information it processed, the actions it took, and the final result it yielded. For more information, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-test.html#trace-events\">Trace enablement</a>.</p>\n </li>\n <li>\n <p>To stream agent responses, make sure that only orchestration prompt is enabled. Agent streaming is not supported for the following steps:\n </p>\n <ul>\n <li>\n <p>\n <code>Pre-processing</code>\n </p>\n </li>\n <li>\n <p>\n <code>Post-processing</code>\n </p>\n </li>\n <li>\n <p>Agent with 1 Knowledge base and <code>User Input</code> not enabled</p>\n </li>\n </ul>\n </li>\n <li>\n <p>End a conversation by setting <code>endSession</code> to <code>true</code>.</p>\n </li>\n <li>\n <p>In the <code>sessionState</code> object, you can include attributes for the session or prompt or, if you configured an action group to return control, results from invocation of the action group.</p>\n </li>\n </ul>\n <p>The response is returned in the <code>bytes</code> field of the <code>chunk</code> object.</p>\n <ul>\n <li>\n <p>The <code>attribution</code> object contains citations for parts of the response.</p>\n </li>\n <li>\n <p>If you set <code>enableTrace</code> to <code>true</code> in the request, you can trace the agent's steps and reasoning process that led it to the response.</p>\n </li>\n <li>\n <p>If the action predicted was configured to return control, the response returns parameters for the action, elicited from the user, in the <code>returnControl</code> field.</p>\n </li>\n <li>\n <p>Errors are also surfaced in the response.</p>\n </li>\n </ul>",
"smithy.api#documentation": "<note>\n </note>\n <p>Sends a prompt for the agent to process and respond to. Note the following fields for the request:</p>\n <ul>\n <li>\n <p>To continue the same conversation with an agent, use the same <code>sessionId</code> value in the request.</p>\n </li>\n <li>\n <p>To activate trace enablement, turn <code>enableTrace</code> to <code>true</code>. Trace enablement helps you follow the agent's reasoning process that led it to the information it processed, the actions it took, and the final result it yielded. For more information, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-test.html#trace-events\">Trace enablement</a>.</p>\n </li>\n <li>\n <p>To stream agent responses, make sure that only orchestration prompt is enabled. Agent streaming is not supported for the following steps:\n </p>\n <ul>\n <li>\n <p>\n <code>Pre-processing</code>\n </p>\n </li>\n <li>\n <p>\n <code>Post-processing</code>\n </p>\n </li>\n <li>\n <p>Agent with 1 Knowledge base and <code>User Input</code> not enabled</p>\n </li>\n </ul>\n </li>\n <li>\n <p>End a conversation by setting <code>endSession</code> to <code>true</code>.</p>\n </li>\n <li>\n <p>In the <code>sessionState</code> object, you can include attributes for the session or prompt or, if you configured an action group to return control, results from invocation of the action group.</p>\n </li>\n </ul>\n <p>The response contains both <b>chunk</b> and <b>trace</b> attributes.</p>\n <p>The final response is returned in the <code>bytes</code> field of the <code>chunk</code> object. The <code>InvokeAgent</code> returns one chunk for the entire interaction.</p>\n <ul>\n <li>\n <p>The <code>attribution</code> object contains citations for parts of the response.</p>\n </li>\n <li>\n <p>If you set <code>enableTrace</code> to <code>true</code> in the request, you can trace the agent's steps and reasoning process that led it to the response.</p>\n </li>\n <li>\n <p>If the action predicted was configured to return control, the response returns parameters for the action, elicited from the user, in the <code>returnControl</code> field.</p>\n </li>\n <li>\n <p>Errors are also surfaced in the response.</p>\n </li>\n </ul>",
"smithy.api#http": {
"code": 200,
"method": "POST",
Expand Down Expand Up @@ -4928,7 +4928,7 @@
}
],
"traits": {
"smithy.api#documentation": "<p>\n Invokes an inline Amazon Bedrock agent using the configurations you provide with the request.\n </p>\n <ul>\n <li>\n <p>Specify the following fields for security purposes.</p>\n <ul>\n <li>\n <p>(Optional) <code>customerEncryptionKeyArn</code> – The Amazon Resource Name (ARN) of a KMS key to encrypt the creation of the agent.</p>\n </li>\n <li>\n <p>(Optional) <code>idleSessionTTLinSeconds</code> – Specify the number of seconds for which the agent should maintain session information. After this time expires, the subsequent <code>InvokeInlineAgent</code> request begins a new session.</p>\n </li>\n </ul>\n </li>\n <li>\n <p>To override the default prompt behavior for agent orchestration and to use advanced prompts, include a <code>promptOverrideConfiguration</code> object. \n For more information, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/advanced-prompts.html\">Advanced prompts</a>.</p>\n </li>\n <li>\n <p>The agent instructions will not be honored if your agent has only one knowledge base, uses default prompts, has no action group, and user input is disabled.</p>\n </li>\n </ul>\n <note>\n <p>The CLI doesn't support streaming operations in Amazon Bedrock, including <code>InvokeInlineAgent</code>.</p>\n </note>",
"smithy.api#documentation": "<p>\n Invokes an inline Amazon Bedrock agent using the configurations you provide with the request.\n </p>\n <ul>\n <li>\n <p>Specify the following fields for security purposes.</p>\n <ul>\n <li>\n <p>(Optional) <code>customerEncryptionKeyArn</code> – The Amazon Resource Name (ARN) of a KMS key to encrypt the creation of the agent.</p>\n </li>\n <li>\n <p>(Optional) <code>idleSessionTTLinSeconds</code> – Specify the number of seconds for which the agent should maintain session information. After this time expires, the subsequent <code>InvokeInlineAgent</code> request begins a new session.</p>\n </li>\n </ul>\n </li>\n <li>\n <p>To override the default prompt behavior for agent orchestration and to use advanced prompts, include a <code>promptOverrideConfiguration</code> object. \n For more information, see <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/advanced-prompts.html\">Advanced prompts</a>.</p>\n </li>\n <li>\n <p>The agent instructions will not be honored if your agent has only one knowledge base, uses default prompts, has no action group, and user input is disabled.</p>\n </li>\n </ul>\n <note>\n </note>",
"smithy.api#http": {
"code": 200,
"method": "POST",
Expand Down Expand Up @@ -5026,6 +5026,12 @@
"traits": {
"smithy.api#documentation": "<p>Model settings for the request.</p>"
}
},
"streamingConfigurations": {
"target": "com.amazonaws.bedrockagentruntime#StreamingConfigurations",
"traits": {
"smithy.api#documentation": "<p>\n Specifies the configurations for streaming.\n </p>\n <note>\n <p>To use agent streaming, you need permissions to perform the <code>bedrock:InvokeModelWithResponseStream</code> action.</p>\n </note>"
}
}
},
"traits": {
Expand Down

0 comments on commit b6204f8

Please sign in to comment.