Skip to content

Commit

Permalink
py-chardet: update to 4.0.0.
Browse files Browse the repository at this point in the history
This will be the last release of chardet to support Python 2.7.
chardet 5.0 will only support 3.6+

Major Changes

This release is multiple years in the making, and provides some
quality of life improvements to chardet. The primary user-facing
changes are:

    Single-byte charset probers now use nested dictionaries under
    the hood, so they are usually a little faster than before. (See
    #121 for details)

    The CharsetGroupProber class now properly short-circuits when
    one of the probers in the group is considered a definite match.
    This lead to a substantial speedup.

    There is now a chardet.detect_all function that returns a list
    of possible encodings for the input with associated confidences.

    We have dropped support for Python 2.6, 3.4, and 3.5 as they
    are all past end-of-life.

The changes in this release have also laid the groundwork for
retraining the models to make them more accurate, and to support
some more encodings/languages (see #99 for progress). This is our
main focus for chardet 5.0 (beyond dropping Python 2 support).
  • Loading branch information
0-wiz-0 committed Jan 4, 2021
1 parent 62400e3 commit 938c4c3
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 11 deletions.
4 changes: 2 additions & 2 deletions converters/py-chardet/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# $NetBSD: Makefile,v 1.26 2020/10/06 10:51:21 wiz Exp $
# $NetBSD: Makefile,v 1.27 2021/01/04 11:53:14 wiz Exp $

DISTNAME= chardet-3.0.4
DISTNAME= chardet-4.0.0
PKGNAME= ${PYPKGPREFIX}-${DISTNAME}
CATEGORIES= converters python
MASTER_SITES= ${MASTER_SITE_PYPI:=c/chardet/}
Expand Down
14 changes: 10 additions & 4 deletions converters/py-chardet/PLIST
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
@comment $NetBSD: PLIST,v 1.9 2017/04/19 17:24:16 wiz Exp $
@comment $NetBSD: PLIST,v 1.10 2021/01/04 11:53:14 wiz Exp $
bin/chardetect-${PYVERSSUFFIX}
${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
Expand Down Expand Up @@ -80,9 +80,6 @@ ${PYSITELIB}/chardet/jpcntx.pyo
${PYSITELIB}/chardet/langbulgarianmodel.py
${PYSITELIB}/chardet/langbulgarianmodel.pyc
${PYSITELIB}/chardet/langbulgarianmodel.pyo
${PYSITELIB}/chardet/langcyrillicmodel.py
${PYSITELIB}/chardet/langcyrillicmodel.pyc
${PYSITELIB}/chardet/langcyrillicmodel.pyo
${PYSITELIB}/chardet/langgreekmodel.py
${PYSITELIB}/chardet/langgreekmodel.pyc
${PYSITELIB}/chardet/langgreekmodel.pyo
Expand All @@ -92,6 +89,9 @@ ${PYSITELIB}/chardet/langhebrewmodel.pyo
${PYSITELIB}/chardet/langhungarianmodel.py
${PYSITELIB}/chardet/langhungarianmodel.pyc
${PYSITELIB}/chardet/langhungarianmodel.pyo
${PYSITELIB}/chardet/langrussianmodel.py
${PYSITELIB}/chardet/langrussianmodel.pyc
${PYSITELIB}/chardet/langrussianmodel.pyo
${PYSITELIB}/chardet/langthaimodel.py
${PYSITELIB}/chardet/langthaimodel.pyc
${PYSITELIB}/chardet/langthaimodel.pyo
Expand All @@ -110,6 +110,12 @@ ${PYSITELIB}/chardet/mbcsgroupprober.pyo
${PYSITELIB}/chardet/mbcssm.py
${PYSITELIB}/chardet/mbcssm.pyc
${PYSITELIB}/chardet/mbcssm.pyo
${PYSITELIB}/chardet/metadata/__init__.py
${PYSITELIB}/chardet/metadata/__init__.pyc
${PYSITELIB}/chardet/metadata/__init__.pyo
${PYSITELIB}/chardet/metadata/languages.py
${PYSITELIB}/chardet/metadata/languages.pyc
${PYSITELIB}/chardet/metadata/languages.pyo
${PYSITELIB}/chardet/sbcharsetprober.py
${PYSITELIB}/chardet/sbcharsetprober.pyc
${PYSITELIB}/chardet/sbcharsetprober.pyo
Expand Down
10 changes: 5 additions & 5 deletions converters/py-chardet/distinfo
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
$NetBSD: distinfo,v 1.9 2017/06/08 19:06:51 adam Exp $
$NetBSD: distinfo,v 1.10 2021/01/04 11:53:14 wiz Exp $

SHA1 (chardet-3.0.4.tar.gz) = 4766fb07e700945a7085d073257f1f320d037ce8
RMD160 (chardet-3.0.4.tar.gz) = 03913482c682bf5e2b872d7f0a25d44fc1df9a47
SHA512 (chardet-3.0.4.tar.gz) = 61a03b23447a2bfe52ceed4dd1b9afdb5784da1933a623776883ee9f297e341f633e27f0ce0230bd5fdc5fdb5382105ab42736a74a417ddeb9f83af57455dba5
Size (chardet-3.0.4.tar.gz) = 1868453 bytes
SHA1 (chardet-4.0.0.tar.gz) = 2384f6cfba4685d901262e073a4455d4cf76d102
RMD160 (chardet-4.0.0.tar.gz) = 55d244d032ebdf38c1da022155699732c9d8e513
SHA512 (chardet-4.0.0.tar.gz) = ebd7f420e1094445270db993f6373ffe7370419e002b0bb13299dc6c9b0f7c4e77b0f44f871fba6371e6869e7c86728514367db377e3137487a3acf50cb81e96
Size (chardet-4.0.0.tar.gz) = 1907771 bytes

0 comments on commit 938c4c3

Please sign in to comment.