Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net Simplify configuration by ServiceId on Multi Model Scenarios. #6416

Merged

Conversation

RogerBarreto
Copy link
Member

Motivation and Context

Setting multiple execution settings is not simple and demands creating a dictionary on the caller side to set directly into the ExecutionSettings setter property.

This change adds a ServiceId property to the execution settings which will be used during the initialization and deserialization to set the expected Key in the dictionary as well as the setting for filtering and executing a service specific function invocation.

With this change were also added new constructors for PromptTemplateConfig, KernelArguments accepting multiple PromptExecutionSettings as well as added multiple for Kernel.CreateFunctionFromPrompt and KernelFunctionFromPrompt.Create

ServiceId Settings

Before:

KernelArguments arguments = [];
arguments.ExecutionSettings = new Dictionary<string, PromptExecutionSettings>()
{
    { serviceId, new PromptExecutionSettings() }
};
var result = await kernel.InvokePromptAsync(prompt, arguments);

After:

var result = await kernel.InvokePromptAsync(prompt, new(new PromptExecutionSettings { ServiceId = serviceId }));

ModelIds Settings

Before:

string[] modelIds = ["model1", "model2", ...];
var modelSettings = new Dictionary<string, PromptExecutionSettings>();
foreach (var modelId in modelIds)
{
    modelSettings.Add(modelId, new PromptExecutionSettings() { ModelId = modelId });
}
var promptConfig = new PromptTemplateConfig(prompt) { Name = "HelloAI", ExecutionSettings = modelSettings };

var function = kernel.CreateFunctionFromPrompt(promptConfig);

After:

string[] modelIds = ["model1", "model2", ...];
var function = kernel.CreateFunctionFromPrompt(prompt, modelIds.Select((modelId, index) => new PromptExecutionSettings { ServiceId = $"service-{index}", ModelId = modelId }));

The same can be done for ServiceId settings:

string[] serviceIds = ["service1", "service2"... ];
var function = kernel.CreateFunctionFromPrompt(prompt, serviceIds.Select(serviceId => new PromptExecutionSettings { ServiceId = serviceId }));

@RogerBarreto RogerBarreto requested a review from a team as a code owner May 27, 2024 21:20
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel kernel.core labels May 27, 2024
@RogerBarreto RogerBarreto self-assigned this May 27, 2024
@markwallace-microsoft
Copy link
Member

Some general feedback

  1. When invoking a prompt a likely use case is that you want to target a specific modelId rather than serviceId. Can you include an example for this if we don't already have one.
  2. If I want to invoke a prompt KernelFunction and specify the modelId or serviceId to use, do we have an example for this?

@RogerBarreto
Copy link
Member Author

@markwallace-microsoft

When invoking a prompt a likely use case is that you want to target a specific modelId rather than serviceId. Can you include an example for this if we don't already have one.

Good point, I added all possible combinations of samples with PreConfigured and per Invocation and simplified a bit more the samples to 2 lines max.

@RogerBarreto RogerBarreto enabled auto-merge June 7, 2024 16:22
@RogerBarreto RogerBarreto added this pull request to the merge queue Jun 19, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 19, 2024
@markwallace-microsoft markwallace-microsoft added this pull request to the merge queue Jun 19, 2024
Merged via the queue into microsoft:main with commit f9a53a1 Jun 19, 2024
15 checks passed
LudoCorporateShark pushed a commit to LudoCorporateShark/semantic-kernel that referenced this pull request Aug 25, 2024
…icrosoft#6416)

### Motivation and Context

Setting multiple execution settings is not simple and demands creating a
dictionary on the caller side to set directly into the
`ExecutionSettings` setter property.

This change adds a `ServiceId` property to the execution settings which
will be used during the initialization and deserialization to set the
expected `Key` in the dictionary as well as the setting for filtering
and executing a service specific function invocation.

With this change were also added new constructors for
`PromptTemplateConfig`, `KernelArguments` accepting multiple
`PromptExecutionSettings` as well as added multiple for
`Kernel.CreateFunctionFromPrompt` and `KernelFunctionFromPrompt.Create`

### ServiceId Settings

Before:

```csharp
KernelArguments arguments = [];
arguments.ExecutionSettings = new Dictionary<string, PromptExecutionSettings>()
{
    { serviceId, new PromptExecutionSettings() }
};
var result = await kernel.InvokePromptAsync(prompt, arguments);
```

After:

```csharp
var result = await kernel.InvokePromptAsync(prompt, new(new PromptExecutionSettings { ServiceId = serviceId }));
```

### ModelIds Settings 

Before:

```csharp
string[] modelIds = ["model1", "model2", ...];
var modelSettings = new Dictionary<string, PromptExecutionSettings>();
foreach (var modelId in modelIds)
{
    modelSettings.Add(modelId, new PromptExecutionSettings() { ModelId = modelId });
}
var promptConfig = new PromptTemplateConfig(prompt) { Name = "HelloAI", ExecutionSettings = modelSettings };

var function = kernel.CreateFunctionFromPrompt(promptConfig);
```

After:

```csharp
string[] modelIds = ["model1", "model2", ...];
var function = kernel.CreateFunctionFromPrompt(prompt, modelIds.Select((modelId, index) => new PromptExecutionSettings { ServiceId = $"service-{index}", ModelId = modelId }));
```

The same can be done for ServiceId settings:

```csharp
string[] serviceIds = ["service1", "service2"... ];
var function = kernel.CreateFunctionFromPrompt(prompt, serviceIds.Select(serviceId => new PromptExecutionSettings { ServiceId = serviceId }));
```

---------

Co-authored-by: Mark Wallace <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel.core kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants