cpython: 3.9.6 -> 3.9.7; 3.8.11 -> 3.8.12; 3.7.11-> 3.7.12; 3.6.14 -> 3.6.15; 3.10.0-rc1 -> 3.10.0

[mweinelt]

Motivation for this change

https://docs.python.org/release/3.9.7/whatsnew/changelog.html#changelog
https://docs.python.org/release/3.8.12/whatsnew/changelog.html#changelog

https://docs.python.org/release/3.6.15/whatsnew/changelog.html#python-3-6-15-final
https://docs.python.org/release/3.7.12/whatsnew/changelog.html#python-3-7-12-final

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• 21.11 Release Notes (or backporting 21.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[mweinelt]

@ofborg build python38.tests python39.tests

[jonringer]

diff LGTM

[jonringer]

LGTM

Result of nixpkgs-review pr 136254 run on x86_64-linux 1

2 packages built:

• python38Packages.requests
• python39Packages.requests

[bergkvist]

By merging this PR - the following issue (on macOS Big Sur) will be resolved: #105038

bpo-44689: ctypes.util.find_library() now works correctly on macOS 11 Big Sur even if Python is built on an older version of macOS. Previously, when built on older macOS systems, find_library was not able to find macOS system libraries when running on Big Sur due to changes in how system libraries are stored.

Both Python 3.8.12 and Python 3.9.7 contain this patch (python/cpython#27251).

[mweinelt]

Two more updates.

python36.tests won't run because our numpy version requires at least python37.

[FRidh]

Note 3.6 will be EOL 2021-12, and will thus need to be removed for NixOS 21.11.

I suggest changing the package to something else than numpy such as py or setuptools.

[jonringer]

Note 3.6 will be EOL 2021-12, and will thus need to be removed for NixOS 21.11.

I suggest changing the package to something else than numpy such as py or setuptools.

Since python36 is essentially deprecated at this point (no longer being recursed into for derivations), I think it's okay to verify that requests still builds, and call it good. Very few people should be using python36 at this point

[jonringer]

unfortunately, python36Packages.requests seems to fail because of a failure in pytest-xdist

[mweinelt]

Marked as security-relevant because of

bpo-43124: Made the internal putcmd function in smtplib sanitize input for presence of \r and \n characters to avoid (unlikely) command injection.

[mweinelt]

Working on removing Python 3.6 now, since we are nearing the release (8 weeks).

pkgs/top-level/all-packages.nix
19735-  doh-proxy = callPackage ../servers/dns/doh-proxy {
19736:    python3Packages = python36Packages;
--
25220-    # see https://github.com/NixOS/nixpkgs/issues/60498
25221:    python3Packages = python36Packages;
--
30589-  caffe2 = callPackage ../development/libraries/science/math/caffe2 (rec {
30590:    inherit (python36Packages) python future six numpy pydot;
30591-    protobuf = protobuf3_1;
30592:    python-protobuf = python36Packages.protobuf.override { inherit protobuf; };

pkgs/applications/misc/plover/default.nix
1:{ lib, fetchurl, python27Packages, python36Packages, wmctrl,
--
26-
27:  dev = with python36Packages; mkDerivationWith buildPythonPackage rec {

pkgs/applications/misc/multibootusb/default.nix
2-  coreutils, gnugrep, which, gnused, e2fsprogs, autoPatchelfHook, gptfdisk,
3:  python36Packages, qt5, runtimeShell, lib, util-linux, wrapQtAppsHook }:
--
15-
16:python36Packages.buildPythonApplication rec {
--
44-    libxcb
45:    python36Packages.python
--
61-  pythonPath = [
62:    python36Packages.dbus-python
63:    python36Packages.pyqt5
64:    python36Packages.pytest-shutil
65:    python36Packages.pyudev
66:    python36Packages.six
--
97-      # Then, add the installed scripts/ directory to the python path
98:      --prefix "PYTHONPATH" ":" "$out/lib/${python36Packages.python.libPrefix}/site-packages"

Wondering if these will cause eval errors. Multibootusb is marked as broken and plover requires python2Packages.pillow, which is marked as insecure.

[mweinelt]

That's it, no more references to python36Packages.

[mweinelt]

@GrahamcOfBorg build python39.tests python38.tests python37.tests

[trofi]

Bisect says d003f75 "python39: 3.9.6 -> 3.9.7" it broke python3Packages.argh tests:

=========================== short test summary info ============================
FAILED test/test_integration.py::test_invalid_choice - assert False
FAILED test/test_integration.py::test_explicit_cmd_name - assert False

_____________________________ test_invalid_choice ______________________________

    def test_invalid_choice():
        def cmd(args):
            return 1

        # root level command

        p = DebugArghParser()
        p.add_commands([cmd])

>       assert run(p, 'bar', exit=True).startswith('invalid choice')
E       assert False
E        +  where False = <built-in method startswith of str object at 0x7fffe89b72d0>('invalid choice')
E        +    where <built-in method startswith of str object at 0x7fffe89b72d0> = "argument {cmd}: invalid choice: 'bar' (choose from 'cmd')".startswith
E        +      where "argument {cmd}: invalid choice: 'bar' (choose from 'cmd')" = run(DebugArghParser(prog='py.test', usage=None, description=None, formatter_class=<class 'argh.constants.CustomFormatter'>, conflict_handler='error', add_help=True), 'bar', exit=True)

test/test_integration.py:380: AssertionError

[trofi]

Filed upstream report as neithere/argh#148

[trofi]

Filed another upstream report at aio-libs/aiohttp#6066

[jktjkt]

There are two for freezegun:

• one due to asyncio, Python 3.10: Deprecation warnings regarding event loop spulec/freezegun#398,
• one about static methods, Python 3.10: TypeError: Tester.helper() takes 0 positional arguments but 1 was given spulec/freezegun#396