Issue with Non-Streaming LLM Responses (Observed with o1 Model)

[PeterDaveHello]

Describe the bug / 描述问题

I've encountered an issue when using the OpenAI o1 model(#958). While the model returns a response, it doesn't seem to be processed and displayed correctly within the application.

Referring to the LangChainBot.js code, the implementation appears to heavily rely on callbacks like handleLLMNewToken. This suggests an expectation of streaming responses from the underlying LLM.

Could this be the reason for the issue observed with the o1 model? Does the current design primarily target streaming scenarios, and is the lack of proper handling for non-streaming responses causing this problem?

To Reproduce / 复现步骤

Checkout code from #958, build and run the app, use the o1 model, and see the empty result. A valid result can be observed from the dev tool.

Expected behavior / 期望行为

LLMs work without streaming: true

Screenshots / 截图

No response

Devtools Info / 开发者工具信息

N/A

OS and version / 操作系统版本

Ubuntu

ChatALL version / ChatALL 版本

main branch

Network / 网络

N/A

Additional context / 其它相关信息

No response

[PeterDaveHello]

@sunner can you help confirm? Thanks.

[sunner]

@PeterDaveHello You are right. The entire project assumes that LLM is working in streaming mode.

I tested #958 by changing o1 to o1-preview and streaming = true;. It looks like the o1 model supports streaming model. Is it necessary to support non-streaming mode?

[PeterDaveHello]

@sunner o1 itself doesn't support strraming yet, that's the reason why, and I dont want to add a preview model at this moment that the GA version was released.

[sunner]

@sunner o1 itself doesn't support strraming yet, that's the reason why, and I dont want to add a preview model at this moment that the GA version was released.

I don't know why o1 model returned a model not found error when I call it. o1-preview works fine. But never mind, I support your decision.

[PeterDaveHello]

I don't know why o1 model returned a model not found error when I call it. o1-preview works fine. But never mind, I support your decision.

o1 currently needs tier 5 usage to access, though OpenAI is spreading the usability.

[sunner]

o1 currently needs tier 5 usage to access, though OpenAI is spreading the usability.

I'm in tier 5. :(

[PeterDaveHello]

o1 currently needs tier 5 usage to access, though OpenAI is spreading the usability.

I'm in tier 5. :(

Use https://platform.openai.com/settings/organization/limits or https://api.openai.com/v1/models to see if o1 is in your model list then, I'm not sure if they offer o1 to ALL tier 5 users yet.