Get ready for Python 2 EOL

[systemcatch]

Python 2 is reaching EOL in 2020 (https://pythonclock.org/).

Me and @jadchaar have chatted about this before but haven't come to a decision yet. We'd hoped that the DST upgrades would be done before the EOL so Python 2.7 users would at least have access to those improvements. However due to being busy we've not had time to work on that.

An important factor in this decision will be how long travis-ci supports 2.7.

dateutil discussion - dateutil/dateutil#653

@hugovk do you have any thoughts or stats on Python 2.7 usage?

[hugovk]

Here's the pip installs for arrow from PyPI for November 2019:

category	percent	downloads
3.7	36.63%	722,467
3.6	24.36%	480,459
2.7	24.06%	474,529
3.5	12.03%	237,263
3.8	1.46%	28,852
null	0.88%	17,421
3.4	0.57%	11,185
2.6	0.00%	86
3.3	0.00%	47
3.9	0.00%	5
Total		1,972,314

Date range: 2019-11-01 - 2019-11-30

Source: pip install -U pypistats && pypistats python_minor arrow --last-month

---

Compare with some other projects:

• https://dev.to/hugovk/python-version-share-over-time-5-7ik

24% is fairly low, and in itself shouldn't be an impediment to dropping.

See also NEP 29, a recommendation for Scientific Python projects to support Python 3.6 and higher:

• https://numpy.org/neps/nep-0029-deprecation_policy.html

If the DST upgrades aren't too far off, no problem in waiting a bit. But I wouldn't wait too long and it might be easier to do them having dropped 2.7.

In general, I'd recommend dropping 2.7 sometime around the EOL date, having made one last release with whatever is unreleased.

[systemcatch]

Thank you Hugo for the stats and analysis which is very helpful.

I was a little suprised at the 24% still on 2.7 but it seems that is similar or below the usage for other major projects.

We will probably release 0.15.5 before Christmas with a warning about Python 2 deprecation and drop support in 0.16.0 on a date to be determined in this issue.

[jadchaar]

I don't know if we necessarily need to emit a warning about Python 2 deprecation because many people are aware that it is reaching EOL in Jan 2020 and it could end up bothering our Python 2 users (throwback to the 0.15.0 warnings).

[hugovk]

And pip has been warning for a long time.

[jaapz]

Could you emit a warning on install, instead of when running?

[systemcatch]

I wasn't thinking of having a warning in the code, more a note in the 0.15.5 changelog about the upcoming deprecation.

[jaapz]

Would maybe dropping 2.7 support warrant a 1.0 release?

[systemcatch]

Errrr maybe, arrow has been available for 7 years after all. The only thing is it adds some extra maintenance burden on me & Jad plus I was really hoping to get DST fixes in before 1.0.0.

[FelixSchwarz]

2 ¢ from a random stranger: Most of my projects did migrate to Python 3 a long time ago but there are (as always) still some important ones using Python 2 (pypy + some enterprise distributions will continue to support Python 2 for quite some time). It would be nice if you could keep Python 2 support unless the maintenance burden becomes too high (e.g. no travis support).
Also I'd appreciate if you could release at least one "major" version with a clear deprecation warning.

[systemcatch]

2 ¢ from a random stranger: Most of my projects did migrate to Python 3 a long time ago but there are (as always) still some important ones using Python 2 (pypy + some enterprise distributions will continue to support Python 2 for quite some time). It would be nice if you could keep Python 2 support unless the maintenance burden becomes too high (e.g. no travis support).
Also I'd appreciate if you could release at least one "major" version with a clear deprecation warning.

@FelixSchwarz while it would be useful for some users, I personally don't want to have to keep thinking about Python 2. Ensuring compatibility and having to backport or miss out on new features in Python 3 all takes time. arrow 0.15.5 is pretty good, many of the longer standing bugs have been ironed out (bar DST) so the version (whatever that is) Python 2 users will be left with should be decent.

Good luck with your projects, I hope you find a way to keep the ones on Python 2 going.

[systemcatch]

Latest download data for April.

category	percent	downloads
3.7	34.55%	849,440
3.6	28.69%	705,355
2.7	19.69%	483,931
3.5	10.34%	254,070
3.8	5.86%	144,129
null	0.57%	14,033
3.4	0.28%	6,851
3.9	0.02%	395
3.3	0.00%	53
2.6	0.00%	52
Total		2,458,309

Date range: 2020-04-01 - 2020-04-30

[systemcatch]

After some internal discussions we have come up with the following plan.

Python 2.7 and 3.5 will be dropped after 2020-09-13 (this is Python 3.5's eol date), this is simpler for us than cutting out 2.7 code only right now.

Also the next version after this drop will be 1.0.0 🥳

[systemcatch]

Python 2.7 & 3.5 Deprecation Checklist

• Update setup.py classifiers and python_requires to declare arrow only supports Python 3.6+
• Drop Python < 3.6 from CI
• Remove Python < 3.6 from Makefile
• Run pyupgrade --py36-plus
• Convert all formatting to f-strings (should be done by pyupgrade)
• Implement type checking
• Remove all future imports
• Remove any remaining python 2 import shadowing
• Audit and overhaul requirements file
• Change timestamp formatting to use datetime.timestamp()
• Replace hardcoded MAX_TIMESTAMP constant with datetime.max.timestamp()
• Convert isinstance(value, numbers.Integral) to isinstance(value, int)
• Remove isstr function
• Remove __cmp__() method in arrow.py
• Remove coverage pragmas and skipped tests for py27

Breaking changes for v1.0.0

• Change timestamp from property to method in arrow.py (requires deprecation warning beforehand)
• Change the return value of Arrow's timestamp method (currently a property) to a float

General Changes for v1.0.0

• Remove beta label from setup.py
• Ensure aligned API between Arrow and datetime in Python 3.6+.

---

Issues related to Py27 and Py35 deprecations

• UnicodeDecodeError while constructing ParserError #177

[krisfremen]

Excellent.

I'm going to start at least some parts to prepare for this.

[jadchaar]

May 2020 stats from pypistats

category	percent	downloads
3.7	33.23%	817,458
3.6	29.29%	720,332
2.7	18.88%	464,420
3.5	9.39%	231,028
3.8	8.14%	200,139
null	0.65%	16,049
3.4	0.39%	9,653
3.9	0.02%	464
3.10	0.00%	108
3.3	0.00%	23
2.6	0.00%	21
Total		2,459,695

Date range: 2020-05-01 - 2020-05-31

[systemcatch]

Rough guide to version usage - https://libraries.io/pypi/arrow/usage

[systemcatch]

@jadchaar @krisfremen stats for June.

category	percent	downloads
3.7	33.26%	830,076
3.6	31.15%	777,558
2.7	15.63%	390,187
3.8	9.69%	241,944
3.5	9.11%	227,414
null	0.76%	18,985
3.4	0.34%	8,401
3.9	0.03%	730
3.10	0.02%	434
3.3	0.00%	67
2.6	0.00%	14
Total		2,495,810

Date range: 2020-06-01 - 2020-06-30

[systemcatch]

Stats for July.

category	percent	downloads
3.6	34.66%	805,401
3.7	30.45%	707,620
2.7	15.81%	367,493
3.8	11.07%	257,144
3.5	6.99%	162,342
null	0.66%	15,245
3.4	0.31%	7,306
3.9	0.04%	1,037
3.10	0.01%	201
3.3	0.00%	49
2.6	0.00%	26
3.2	0.00%	2
Total		2,323,866

Date range: 2020-07-01 - 2020-07-31