Python 3.5 Deprecation Checklist / Chores

[callahad]

We've announced the deprecation of Python 3.5, intending to break compatibility no sooner than April 2021.

So here are a bunch of small tasks we can take care of when April rolls around:

• Bump the version of Black we use for code formatting
• Bump the tool.black.target-version in pyproject.toml
• Run Black on all the things
• Update our Trove classifiers to drop support for 3.5
• Update CI configuration to stop testing on 3.5
• Update .deb build script to stop building for Debian Stretch and Ubuntu Xenial
• Drop the weird build script bits around dh-systemd
• Drop the dependency constraints for pyjwt / pysaml
• Audit our python package's extras -- I think I recall something in [lint] not supporting 3.5, and that complicating install / contributing docs, maybe?
• Announce to devs that we can finally use f'strings'
• Py 3.6 marked asyncio's API as stable; check if there's anywhere that would make life easier for us
• Py 3.6 introduced a new secrets module to replace os.urandom, see if that would make life easier for us
• Grep source and docs for anything referencing Python 3.5, Stretch, or Xenial and excise it
• Update the minimum version of unpaddedbase64 to 2.1.0 and remove its ignore_missing_imports line from mypy.ini

[ShadowJonathan]

When 3.5 compatibility is finally dropped, i'll tackle #8332 by using pyupgrade --py36-plus on the codebase, this'll upgrade a bunch more code to use pythonic 3.6 code (in a similar matter to black)

Py 3.6 marked asyncio's API as stable; check if there's anywhere that would make life easier for us

Synapse is head-and-toes-deep into twisted, if you'd like to formally announce to move to asyncio, that'd be a huge boon on far-in-the-future compatibility, imo. In the meantime, it'd likely be very possible to just start using the asyncio twisted reactor, which makes it possible to mix in asyncio-based libraries, which can make a bunch of our lives easier with using more modern frameworks/methods/libraries, or indeed starting that slow transition.

Diversion: I saw twisted use "fade to black" as the commit that massively black-ed their codebase, maybe use that for this as well? 😛

[ShadowJonathan]

• Extra chore: Simplify this snippet

[ShadowJonathan]

I think we'll keep this snippet to make sure people actually don't run synapse on 3.5? (as unlikely as that is possible when we start adding more 3.6 syntax changes)

Py 3.6 introduced a new secrets module to replace os.urandom, see if that would make life easier for us

• That affects this snippet

[ShadowJonathan]

com2ann (gh) looks like a good way to also convert any comment-type-annotations to inline type annotations, I ran it on the synapse codebase, and mypy seems to complain a lot about None assignments to typed variables, this concerns me a bit, so maybe it's worth looking into.

3.6 allows type annotations on variables, thus the suggestion.

[callahad]

Hynek is a big 👍 on pyupgrade, too: https://twitter.com/hynek/status/1357651722852786176

I COULD say that I don’t learn about cool new Python features by running pyupgrade over my repos.

But then I would be lying.

[aaronraimist]

• Update INSTALL.md which still states "Matrix.org provides Debian/Ubuntu packages of the latest stable version of Synapse via https://packages.matrix.org/debian/. They are available for Debian 9 (Stretch), Ubuntu 16.04 (Xenial), and later."

https://github.com/matrix-org/synapse/blob/develop/INSTALL.md#matrixorg-packages

[clokep]

I just checked off a few more tasks on here. There's a couple more:

Audit our python package's extras -- I think I recall something in [lint] not supporting 3.5, and that complicating install / contributing docs, maybe?

I didn't see anything about this, but not sure if I missed it.

Py 3.6 marked asyncio's API as stable; check if there's anywhere that would make life easier for us

I don't think there is much here, honestly.

I suspect this can be closed. What do you think @callahad?

[callahad]

I suspect this can be closed. What do you think @callahad?

Sure; time to open a new one for dropping 3.6, too :)