-
-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make chardet optional or use charset-normalizer instead #222
Comments
chardet is not required in aeidon/gaupol, it's imported only under the I think I was a bit eager to make dependencies optional back when writing these. Encoding auto-detection is something that probably 95% of users want, since they're downloading random subtitle files from the internet and can't really know the encoding. |
If you do Anyway, I've created #223 to migrate to charset-normalizer, please review it if this is something you want. |
While memory profiling Weblate I've noticed that 2+ MB is consumed on
chardet
module which we directly don't depend on.The only reverse dependency for
chardet
isgaupol
in our case. Everybody else seems to have switched tocharset-normalizer
instead, which is a maintained, faster and low memory footprint alternative.I'm willing to contribute a pull request, but first I'd need to know which direction you prefer. Two approaches I can see:
charset-normalizer
, but it supports only Python 3.5 and newer while aeidon supports older Python versions, so it would mean raising the bar. I don't think this should be an issue these days.chardet
from required dependencies, moving it to extras.The text was updated successfully, but these errors were encountered: