Preparing rc1 release.

bindings/node/CHANGELOG.md

@@ -1,3 +1,8 @@
+## [0.13.0] 
+
+- [#1008] `Decoder` is now a composable trait, but without being backward incompatible
+- [#1047, #1051, #1052] `Processor` is now a composable trait, but without being backward incompatible
+
 ## [0.12.1] 
 
 - [#938] **Reverted breaking change**. https://github.com/huggingface/transformers/issues/16520
@@ -160,6 +165,13 @@ The files must now be provided first when calling `tokenizer.train(files, traine
 - Actually add special tokens in tokenizers implementations ([acef252](https://github.com/huggingface/tokenizers/commit/acef252dacc43adc414175cfc325668ad1488753))
 
 
+[#956]: https://github.com/huggingface/tokenizers/pull/956
+[#1008]: https://github.com/huggingface/tokenizers/pull/1008
+[#1009]: https://github.com/huggingface/tokenizers/pull/1009
+[#1047]: https://github.com/huggingface/tokenizers/pull/1047
+[#1055]: https://github.com/huggingface/tokenizers/pull/1055
+[#1051]: https://github.com/huggingface/tokenizers/pull/1051
+[#1052]: https://github.com/huggingface/tokenizers/pull/1052
 [#938]: https://github.com/huggingface/tokenizers/pull/938
 [#939]: https://github.com/huggingface/tokenizers/pull/939
 [#952]: https://github.com/huggingface/tokenizers/pull/952

bindings/node/package.json

@@ -1,6 +1,6 @@
 {
     "name": "tokenizers",
-    "version": "0.12.1",
+    "version": "0.13.0",
     "description": "",
     "main": "./dist/index.js",
     "types": "./dist/index.d.ts",

bindings/python/CHANGELOG.md

@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.13.0] 
+
+- [#956] PyO3 version upgrade
+- [#1055] M1 automated builds
+- [#1008] `Decoder` is now a composable trait, but without being backward incompatible
+- [#1047, #1051, #1052] `Processor` is now a composable trait, but without being backward incompatible
+
+Both trait changes warrant a "major" number since, despite best efforts to not break backward
+  compatibility, the code is different enough that we cannot be exactly sure.
+
 ## [0.12.1] 
 
 - [#938] **Reverted breaking change**. https://github.com/huggingface/transformers/issues/16520
@@ -376,6 +386,13 @@ delimiter (Works like `.split(delimiter)`)
 - Fix a bug that was causing crashes in Python 3.5
 
 
+[#956]: https://github.com/huggingface/tokenizers/pull/956
+[#1008]: https://github.com/huggingface/tokenizers/pull/1008
+[#1009]: https://github.com/huggingface/tokenizers/pull/1009
+[#1047]: https://github.com/huggingface/tokenizers/pull/1047
+[#1055]: https://github.com/huggingface/tokenizers/pull/1055
+[#1051]: https://github.com/huggingface/tokenizers/pull/1051
+[#1052]: https://github.com/huggingface/tokenizers/pull/1052
 [#938]: https://github.com/huggingface/tokenizers/pull/938
 [#939]: https://github.com/huggingface/tokenizers/pull/939
 [#952]: https://github.com/huggingface/tokenizers/pull/952

bindings/python/py_src/tokenizers/__init__.py

@@ -1,4 +1,4 @@
-__version__ = "0.12.1.dev0"
+__version__ = "0.13.0.dev0"
 
 from typing import Tuple, Union, Tuple, List
 from enum import Enum

bindings/python/setup.py

@@ -8,7 +8,7 @@
 
 setup(
     name="tokenizers",
-    version="0.12.1.dev0",
+    version="0.13.0.dev0",
     description="Fast and Customizable Tokenizers",
     long_description=open("README.md", "r", encoding="utf-8").read(),
     long_description_content_type="text/markdown",

tokenizers/CHANGELOG.md

@@ -4,6 +4,15 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.13.0] 
+
+- [#1009] `unstable_wasm` feature to support building on Wasm (it's unstable !)
+- [#1008] `Decoder` is now a composable trait, but without being backward incompatible
+- [#1047, #1051, #1052] `Processor` is now a composable trait, but without being backward incompatible
+
+Both trait changes warrant a "major" number since, despite best efforts to not break backward
+  compatibility, the code is different enough that we cannot be exactly sure.
+
 ## [0.12.1] 
 
 - [#938] **Reverted breaking change**. https://github.com/huggingface/transformers/issues/16520
@@ -157,6 +166,13 @@ split up in multiple bytes
 - [#174]: The `LongestFirst` truncation strategy had a bug
 
 
+[#956]: https://github.com/huggingface/tokenizers/pull/956
+[#1008]: https://github.com/huggingface/tokenizers/pull/1008
+[#1009]: https://github.com/huggingface/tokenizers/pull/1009
+[#1047]: https://github.com/huggingface/tokenizers/pull/1047
+[#1055]: https://github.com/huggingface/tokenizers/pull/1055
+[#1051]: https://github.com/huggingface/tokenizers/pull/1051
+[#1052]: https://github.com/huggingface/tokenizers/pull/1052
 [#938]: https://github.com/huggingface/tokenizers/pull/938
 [#939]: https://github.com/huggingface/tokenizers/pull/939
 [#952]: https://github.com/huggingface/tokenizers/pull/952

tokenizers/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Anthony MOI <[email protected]>"]
 edition = "2018"
 name = "tokenizers"
-version = "0.12.1"
+version = "0.13.0"
 homepage = "https://github.com/huggingface/tokenizers"
 repository = "https://github.com/huggingface/tokenizers"
 documentation = "https://docs.rs/tokenizers/"