Skip to content

Commit

Permalink
Fixed the warning message "Setting pad_token_id to eos_token_id:1… (
Browse files Browse the repository at this point in the history
ShishirPatil#110)

When running the falcon model, after the user prompt, the model response
is prefixed with this warning.
```
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results. Setting `pad_token_id` to `eos_token_id`:11 for open-end generation. Fix for this issue
```

So Provided a fix.
  • Loading branch information
dineshkumarsarangapani authored Jul 17, 2024
1 parent 16af6c8 commit fe76692
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions inference/serve/gorilla_falcon_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ def get_response(prompt, model, tokenizer, device):
do_sample=True,
temperature=0.7,
max_new_tokens=1024,
pad_token_id=tokenizer.eos_token_id
)
output_ids = output_ids[0][len(input_ids[0]) :]
outputs = tokenizer.decode(output_ids, skip_special_tokens=True).strip()
Expand Down

0 comments on commit fe76692

Please sign in to comment.