[Bug]: gpt-4o and gpt-4o-mini models not working for assistants

[stuart-pebble]

Language

Javascript/Typescript

Version

latest

Description

When using OpenAI Assistants, only some models work, not all of them. None of the gpt4o variants work. Its worth noting that gpt-4-1106-preview and gpt-4-turbo models do work.

Reproduction Steps

1. Build an MS Teams Assistant API bot using the defaults 
2. Modify the OpenAI Assistant to use the preferred model
3.
...

[corinagum]

Hi @stuart-pebble, which sample are you using? Order bot? or Math bot? What errors are you getting?

[stuart-pebble]

I'm not using a sample, I'm using the MS Teams Toolkit to create one from scratch using:

Create New App

Custom Copilot:

AI Agent:

Build with Assistants API (Preview)

JavaScript

My objective is a simple RAG use case where users can ask questions with responses based on RAG from an OpenAI configured vector store. I can work around the gpt-4o-mini issue and select the gpt-4-turbo model in the short term but its not optimal.

FYI: I'm also having the same issues as #1585 where, using the defaults, the OpenAI RAG functions are ignored. I'm currently exploring options to either update the node module or redeploy using the orderbot example.

[corinagum]

@stuart-pebble what errors are you getting? What is your sample code? We will need information in order to determine the bug.

[stuart-pebble]

If I configure the Assistant in OpenAI with gpt-4o or gpt-4o-mini all I see in the MS Teams chabot is " There was an error". If I change to one of the other models it just works.

[lariar]

Came here to confirm also seeing this problem. Took 3 days to figure out my own chatbot would work by reverting it back to GPT-4-turbo on the OpenAI side.

[yikwen]

I faced this issue as well where older GPT models worked fine and things started breaking when changed to GPT-4o. Inspecting the completions coming from GPT-4o, I found that the issue was with GPT-4o not following instructions in creating the plan using SAY and DO commands (I'm using sequence augmentation btw).

I added the following sentences to skprompt.txt and it performed better subsequently.

• Provide your response in the form of a JSON object following the format provided at the end of these instructions
• If the user's request matches an action you can perform, add that action to the JSON plan as a DO command. For anything you want to ask or tell the user, add the text to the JSON plan as a SAY command.

[Stevenic]

Thanks for the hint @yikwen. We're going to look into using the new structured outputs support for plan generation which should help resolve the issue.

[Stevenic]

@stuart-pebble I think the issue you're hitting is something different so I'll try to reproduce it

[Benjiiim]

We're going to look into using the new structured outputs support for plan generation which should help resolve the issue.

The perspective to have Teams AI Library using structured outputs and GPT-4o-mini is really exciting! I can't wait to see the impact it will have, especially on cost-effectiveness.
Is this something that we can expect to get our hands on soon?

[singhk97]

If you're using the JS SDK then the underlying client of the AssistantPlanner does not pass the beta-v2 tag to Assistants API calls which is required to use the gpt-4o models.

[aacebo]

closing this since its related to the assistants v2 work that is being tracked in another ticket. If interested please refer to #1585 for updates.