Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 bug: DeepSeek R1 model integration and adaptation issues. #297

Open
akinlong opened this issue Feb 13, 2025 · 1 comment
Open

🐛 bug: DeepSeek R1 model integration and adaptation issues. #297

akinlong opened this issue Feb 13, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@akinlong
Copy link

akinlong commented Feb 13, 2025

Description

I am trying to use DeepSeek R1 as backend LLM service of MM AI copilot, it fails when calling the backend API, the server logs shows:
message: invalid character '<' looking for beginning of value

As we know, the DeepSeek R1 response starts with <think> part, some AI client tools will display the 'think' part as a single breakdown UI.
My question is, do we have plans to fix the reason model or fully support the reason model in the future?

Steps to reproduce

Config DeepSeek R1 model as backup back AI service and post a message to the copilot.

@akinlong akinlong added the bug Something isn't working label Feb 13, 2025
@akinlong
Copy link
Author

akinlong commented Feb 13, 2025

My mistake, standard DS API doesn't return think part, it was designed by qwen.

After I switched to the latest version of VLLM and re-run the DeepSeek R1 model, the copilot is now able to properly accept the LLM's output. However, the reasoning part still experiences lag in the UI and does not display the reasoning process. Will this feature be implemented in the UI in the future?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant