Revert "[BREAKING CHANGE] Ignore added_tokens (both special and not) … (

#1569)

* Revert "[BREAKING CHANGE] Ignore added_tokens (both special and not) in the decoder (#1513)"

This reverts commit 25aee8b.

* don't remove audit

* deprecate id_to_token

* use simple id to token

* don't break id_to_token since we are deprecating anyways?

tokenizers/src/tokenizer/mod.rs

@@ -849,7 +849,8 @@ where
             .iter()
             .filter_map(|id| {
                 self.added_vocabulary
-                    .id_to_token(*id, &self.model)
+                    .simple_id_to_token(*id)
+                    .or_else(|| self.model.id_to_token(*id))
                     .filter(|token| {
                         !skip_special_tokens || !self.added_vocabulary.is_special_token(token)
                     })