diff --git a/getting-started/CONCEPTS/STREAMING.md b/getting-started/CONCEPTS/STREAMING.md index a9894fcb9..c7bd293da 100644 --- a/getting-started/CONCEPTS/STREAMING.md +++ b/getting-started/CONCEPTS/STREAMING.md @@ -42,7 +42,7 @@ There are three parts to streaming: - [Python Streaming+Tools LightBot](https://github.com/microsoft/teams-ai/tree/main/python/samples/04.ai.c.actionMapping.lightBot) ## Streaming Response Class -The `StreamingResponse` class is the helper class for streaming responses to the client. The class is used to send a series of updates to the client in a single response. If you are using your own custom model, you can directly instantiate and manage this class to stream responses. +The `StreamingResponse` class is the helper class for streaming responses to the client. The class is used to send a series of updates to the client in a single response. The expected sequence of calls is: @@ -52,6 +52,8 @@ The expected sequence of calls is: Once `endStream()` is called, the stream is considered ended and no further updates can be sent. +If you are using your own custom model, you can directly instantiate and manage this class to stream responses. The instructions for this scenario are specified below, though we encourage you to read through this to its entirety to understand the complete flow. + ## Configuration with Azure Open AI / Open AI @@ -208,6 +210,77 @@ planner=ActionPlanner( ``` +## Custom `StreamingResponse` Management + +### Definitions + +- **Stream ID**: A unique identifier for the stream, assigned after the initial update is sent. It ensures that all subsequent messages are associated with the correct stream. +- **Sequence Numbers**: Each message in the stream is assigned a sequence number (`streamSequence`), starting from 1 and incrementing with each message. This helps maintain the order of messages. + + +The class also maintains an internal queue for outgoing activities to ensure they are sent in order and without overloading the Teams client. + +### Implementation + +#### 1. Initialize `StreamingResponse` + +To start using the `StreamingResponse` class, you need to create an instance of it within your bot's turn handler. + + ```javascript + const streamingResponse = new StreamingResponse(context); + ``` + + +#### 2. Send Informative Updates + + ```javascript + streamingResponse.queueInformativeUpdate('Searching for the information you requested...'); + ``` + +#### 3. Queuing Text Chunks + +As your bot generates or retrieves content, you can send partial messages to the user. + +**Note 1**: As we do not handle chunking, this class assumes you already have some form of chunking mechanism (e.g., via another LLM) prior to using this class. +The streaming behaviour is determined through the timeouts and the size of the text chunks. + +**Note 2**: It is important to include a delay between the chunks to avoid spamming the Teams Client. + + ```javascript + const firstChunk = 'Here is the first part of the information.'; + const secondChunk = 'Continuing with more details...'; + + streamingResponse.queueTextChunk(firstChunk); + + // Delay between chunks, minimum 1000ms + await new Promise(resolve => setTimeout(resolve, 1500)); + + streamingResponse.queueTextChunk(secondChunk); + ``` + +#### 4. Ending the Stream + +Call the `endStream` method to conclude the stream. Since this method returns a promise, use `await` to ensure all queued activities are sent before proceeding. + + + ```javascript + await streamingResponse.endStream(); + ``` + +### Extra Features + +You may also be interested in adding in the [Powered by AI](./POWERED-BY-AI.md) features. + +To configure the Feedback Loop, use the `setFeedbackLoop` and `setFeedbackLoopType` methods. + +To configure the Generated by AI button, use the `setGeneratedByAILabel` method. + +To configure the Sensitivity Label, use the `setSensitivityLabel` method. + +To configure citations, use the `setCitations` method. Please see the ClientCitation interface in the Teams AI library for the expected shape of the citation object. + +To configure attachments, use the `setAttachments` method. Note that this is only currently rendered for the final chunk. + --- ## Return to other major section topics: