Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to Python 3 by default #6647

Closed
brandjon opened this issue Nov 9, 2018 · 4 comments
Closed

Move to Python 3 by default #6647

brandjon opened this issue Nov 9, 2018 · 4 comments
Assignees
Labels
P1 I'll work on this now. (Assignee required) team-Rules-Python Native rules for Python type: feature request

Comments

@brandjon
Copy link
Member

brandjon commented Nov 9, 2018

Once #6583 is implemented, we'll want to look at switching the default Python version to PY3. The default version is the version used when no version selector attribute is explicitly set on a py_binary or via a top-level flag.

We can do this change by adding an incompatible flag, e.g. --incompatible_default_python_version_is_py3.

Another possibility is to also add an extra flag, --default_python_version, that could be either PY2 or PY3 (but with the incompatible flag on it would itself default to PY3). This would allow whole projects to customize the default value indefinitely. I'm not sure whether this is useful. In fact I think it could be harmful, because it means the semantics of Python targets in deep dependencies (including in external repos) could change based on the top-level flag.

@brandjon brandjon added type: feature request P2 We'll consider working on this in future. (Assignee optional) team-Rules-Python Native rules for Python labels Nov 9, 2018
@brandjon
Copy link
Member Author

brandjon commented Feb 4, 2019

Prioritizing. We want this to be available in --all_incompatible_changes starting in 0.24, probably to be flipped in 0.25. Current plan is as follows:

  1. Create a flag --default_python_version=[PY2|PY3], with default value of PY2. When --incompatible_allow_python_version_transitions is enabled (the new version semantics), this flag determines the python version to use for any py_binary/py_test target that does not specify a python_version (previously called default_python_version) attribute. When --incompatible_allow_python_version_transitions is disabled, this flag has no effect.

  2. Create a flag --incompatible_py3_is_default. When enabled, this changes the default value of --default_python_version from PY2 to PY3.

  3. Create a flag --incompatible_py2_outputs_are_suffixed. When enabled, PY2 artifacts will be placed in an output directory whose name includes a "-py2" suffix, and PY3 artifacts will be placed in a directory without a Python-related suffix (typically pointed to by the bazel-out symlink). This is the opposite of what happens today, where PY2 artifacts have no suffix / can be bazel-out, but PY3 targets have a "-py3" suffix.

We want to change the output path naming because it'd be weird if targets aren't put in bazel-out by default. Technically, users aren't supposed to rely on the specifics of the output path naming, but in practice people do, hence the need for the incompatible change flag.

@brandjon brandjon added P1 I'll work on this now. (Assignee required) and removed P2 We'll consider working on this in future. (Assignee optional) labels Feb 4, 2019
@brandjon brandjon self-assigned this Feb 4, 2019
@brandjon
Copy link
Member Author

brandjon commented Feb 4, 2019

I should say, --default_python_version will also set the python version for host configuration, unless --host_force_python overrides it.

@brandjon
Copy link
Member Author

brandjon commented Feb 6, 2019

So on further reflection, I think it is indeed problematic to indefinitely allow customizing the default version via a flag, since (as I pointed out above) it makes it hard to compose different repos in one build. I'm considering scrapping --default_python_version and just having --incompatible_py3_is_default take its place in directly affecting the default version.

bazel-io pushed a commit that referenced this issue Feb 7, 2019
…t`'s default version

When this flag is enabled, `py_binary` and `py_test` targets that do not specify a version (by setting the `python_version` attribute, or its deprecated alias `default_python_version`) will default to `PY3` instead of `PY2`. In addition, the host configuration will use `PY3`, unless `--host_force_python=PY2` is set to override it.

The flag requires the new version semantics (`--incompatible_allow_python_version_transitions=true`), and will fail the build if it is not set.

The flag is implemented by eliminating PythonVersion#DEFAULT_TARGET_VALUE and replacing it with an accessor PythonOptions#getDefaultPythonVersion. For the case of rule transition factories in rule definitions, which previously relied on DEFAULT_TARGET_VALUE and which do not have access to a configuration, they are updated to instead pass a callback function that retrieves the default version from the configuration at the time of the transition.

Work toward #6647, #7359.

RELNOTES[INC]: Added --incompatible_py3_is_default to test switching the default Python version to PY3 for py_binary/py_test targets that do not specify a version. See #7359.

PiperOrigin-RevId: 232939284
bazel-io pushed a commit that referenced this issue Feb 28, 2019
This adds --incompatible_py2_outputs_are_suffixed, which makes it so Python 2 targets get built under an output root with the "-py2" suffix, and Python 3 targets get built under a root with no additional suffix. This means that the `bazel-bin` symlink contains Python 3 targets instead of Python 2 targets.

This goes hand-in-hand with --incompatible_py3_is_default, which controls whether targets get built in the Python 2 or Python 3 configuration by default. Enabling just one flag or the other will cause the majority of targets to get built under an output root other than the one pointed to by `bazel-bin`, which is not very convenient.

In general users and scripts are not supposed to care what the exact output path of a target is, but in practice they do, hence why this is an incompatible change.

Work toward #6647. See #7593 for migration info.

RELNOTES[INC]: Added --incompatible_py2_outputs_are_suffixed, for switching the bazel-bin symlink to point to Python 3 outputs instead of Python 2 outputs. See [#7593](#7593).

PiperOrigin-RevId: 236207234
@brandjon
Copy link
Member Author

According to downstream presubmit results, we'll be able to flip both flags for 0.25.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-Rules-Python Native rules for Python type: feature request
Projects
None yet
Development

No branches or pull requests

1 participant