You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
For models without a tokenizer, e.g. Othello, an error is thrown in the ActivationsStore since it assumes a tokenizer is present and attempts to insert BOS tokens. If the model has no Tokenizer, we should gracefully handle this and not attempt to read anything form the tokenizer.
Describe the bug
For models without a tokenizer, e.g. Othello, an error is thrown in the ActivationsStore since it assumes a tokenizer is present and attempts to insert BOS tokens. If the model has no Tokenizer, we should gracefully handle this and not attempt to read anything form the tokenizer.
Code example
See #177
Checklist
The text was updated successfully, but these errors were encountered: