Code

Simplify lang detection#299

Code

Merged

meili-bors[bot] merged 8 commits intomainmeilisearch/charabia:mainfrom simplify-lang-detectionmeilisearch/charabia:simplify-lang-detectionCopy head branch name to clipboardJul 25, 2024

+139-107

Commits

Commits on Jul 10, 2024

• Simplify lang detection

  ManyTheFish

  committed
  28d1305

  Copy full SHA for 28d1305

Commits on Jul 11, 2024

• Remove Charabia Language layer

  ManyTheFish

  committed
  a7d34a1

  Copy full SHA for a7d34a1

Commits on Jul 22, 2024

• Allow to dynamicaly change the language allow list when tokenizing a text

  ManyTheFish

  committed
  f9732b2

  Copy full SHA for f9732b2

• Fix tests

  ManyTheFish

  committed
  599bb3b

  Copy full SHA for 599bb3b

Commits on Jul 23, 2024

• Update charabia/src/tokenizer.rs

  Show description for a95a921
  

  ManyTheFish

  and

  dureuill

  authored
  a95a921

  Copy full SHA for a95a921

• fix

  ManyTheFish

  committed
  357e53b

  Copy full SHA for 357e53b

• Remove into_option method

  ManyTheFish

  committed
  2992583

  Copy full SHA for 2992583

Commits on Jul 24, 2024

• Update charabia/src/tokenizer.rs

  ManyTheFish

  authored
  cbeb12a

  Copy full SHA for cbeb12a