Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(model gallery): update deepseek-r1 prompt template #4686

Merged
merged 1 commit into from
Jan 25, 2025

Conversation

M0Rf30
Copy link
Collaborator

@M0Rf30 M0Rf30 commented Jan 25, 2025

Description

This PR fixes deepseek-r1 prompt. </think> splitting is missing, I already tried with cutstrings parameter but seems that regex doesn't work.

Notes for Reviewers
This is the entire jinja template

# {% if not add_generation_prompt is defined %}
#   {% set add_generation_prompt = false %}
# {% endif %}
# {% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true) %}
# {%- for message in messages %}
#   {%- if message['role'] == 'system' %}
#     {%- if ns.is_first_sp %}
#       {% set ns.system_prompt = ns.system_prompt + message['content'] %}
#       {% set ns.is_first_sp = false %}
#     {%- else %}
#       {% set ns.system_prompt = ns.system_prompt + '\\n\\n' + message['content'] %}
#     {%- endif %}
#   {%- endif %}
# {%- endfor %}
# {{ bos_token }}{{ ns.system_prompt }}
# {%- for message in messages %}
#   {%- if message['role'] == 'user' %}
#     {%- set ns.is_tool = false -%}
#     {{'<|User|>' + message['content']}}
#   {%- endif %}
#   {%- if message['role'] == 'assistant' and 'tool_calls' in message %}
#     {%- set ns.is_tool = false -%}
#     {%- for tool in message['tool_calls'] %}
#       {%- if not ns.is_first %}
#         {%- if message['content'] is none %}
#           {{'<|Assistant|><|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}
#         {%- else %}
#           {{'<|Assistant|>' + message['content'] + '<|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}
#         {%- endif %}
#         {%- set ns.is_first = true -%}
#       {%- else %}
#         {{'\\n' + '<|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}
#       {%- endif %}
#     {%- endfor %}
#   {{'<|tool▁calls▁end|><|end▁of▁sentence|>'}}
#   {%- endif %}
#   {%- if message['role'] == 'assistant' and 'tool_calls' not in message %}
#     {%- if ns.is_tool %}
#       {{'<|tool▁outputs▁end|>' + message['content'] + '<|end▁of▁sentence|>'}}
#       {%- set ns.is_tool = false -%}
#     {%- else %}
#       {% set content = message['content'] %}
#       {% if '</think>' in content %}
#         {% set content = content.split('</think>')[-1] %}
#       {% endif %}
#       {{'<|Assistant|>' + content + '<|end▁of▁sentence|>'}}
#     {%- endif %}
#   {%- endif %}
#   {%- if message['role'] == 'tool' %}
#     {%- set ns.is_tool = true -%}
#     {%- if ns.is_output_first %}
#       {{'<|tool▁outputs▁begin|><|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}
#       {%- set ns.is_output_first = false %}
#     {%- else %}
#       {{'<|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}
#     {%- endif %}
#   {%- endif %}
# {%- endfor -%}
# {% if ns.is_tool %}
#   {{'<|tool▁outputs▁end|>'}}
# {% endif %}
# {% if add_generation_prompt and not ns.is_tool %}
#   {{'<|Assistant|>'}}
# {% endif %}

Signed commits

  • Yes, I signed my commits.

Copy link

netlify bot commented Jan 25, 2025

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 217bb2e
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/679448d9b33cb20008a6824a
😎 Deploy Preview https://deploy-preview-4686--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
Owner

@mudler mudler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good here! 🙏 Thanks!

@mudler mudler merged commit e9cace1 into mudler:master Jan 25, 2025
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants