Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve ChatML template #5411

Merged
merged 29 commits into from
Feb 5, 2024
Merged

Improve ChatML template #5411

merged 29 commits into from
Feb 5, 2024

Conversation

BadisG
Copy link
Contributor

@BadisG BadisG commented Jan 31, 2024

Hello,

The ChatML template must look like this:

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

But the current ChatML template gives this result

<|im_start|>system
{system_message}
<|im_end|>
<|im_start|>user
{prompt}
<|im_end|>
<|im_start|>assistant

This can lead to really incoherent outputs when talking to the bot.
I've also tweaked the code a bit so that when there's no system prompt, this disappears:

<|im_start|>system
<|im_end|>

oobabooga and others added 29 commits December 14, 2023 22:39
@oobabooga
Copy link
Owner

That seems correct, thanks

@oobabooga oobabooga changed the base branch from main to dev February 5, 2024 02:39
@oobabooga oobabooga merged commit 9fdee65 into oobabooga:dev Feb 5, 2024
PoetOnTheRun pushed a commit to PoetOnTheRun/text-generation-webui that referenced this pull request Feb 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants