Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ruff] Implement redundant-bool-literal (RUF038) #14319

Merged
merged 8 commits into from
Nov 16, 2024

Conversation

sbrugman
Copy link
Contributor

@sbrugman sbrugman commented Nov 13, 2024

Summary

Implements redundant-bool-literal

Test Plan

cargo test

The ecosystem results are all correct, but for Airflow the rule is not relevant due to the use of overloading (and is marked as unsafe correctly).

Copy link
Contributor

github-actions bot commented Nov 13, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+8 -0 violations, +0 -0 fixes in 4 projects; 50 projects unchanged)

apache/airflow (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ airflow/models/dagrun.py:1317:23: RUF038 `Literal[True, False]` can be replaced with `bool`
+ airflow/models/dagrun.py:1439:23: RUF038 `Literal[True, False]` can be replaced with `bool`

pandas-dev/pandas (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ pandas/core/dtypes/dtypes.py:1271:15: RUF036 `None` not at the end of the type annotation.

python/typeshed (+4 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select E,F,FA,I,PYI,RUF,UP,W

+ stdlib/ast.pyi:1480:16: RUF038 `Literal[True, False]` can be replaced with `bool`
+ stdlib/ast.pyi:1481:35: RUF038 `Literal[True, False]` can be replaced with `bool`
+ stdlib/ast.pyi:1484:45: RUF038 `Literal[True, False]` can be replaced with `bool`
+ stubs/pyxdg/xdg/Menu.pyi:97:11: RUF038 `Literal[True, False, ...]` can be replaced with `Literal[...] | bool`

wntrblm/nox (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ nox/command.py:33:16: RUF038 `Literal[True, False, ...]` can be replaced with `Literal[...] | bool`

Changes by rule (2 rules affected)

code total + violation - violation + fix - fix
RUF038 7 7 0 0 0
RUF036 1 1 0 0 0

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you and I especially like the detailed fix safety documentation. @AlexWaygood would you mind signing off the rule?

@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Nov 14, 2024
@MichaReiser
Copy link
Member

What do you think of true-false-literal as a rule name? Because the first thing I think of when reading redundant is that the rule is about Literal[True, True] when it isn't

@sbrugman
Copy link
Contributor Author

I've got no strong naming preference. I considered true-false, but then Literal[True, False, True, False] are also detected. The criterium is that the values in the literal are exhaustive for the type.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, how common is this antipattern in practice? I see a few ecosystem hits here, but not enough to make me think that this is something that a lot of users are doing. Has this rule been requested by any of our userse?

For context: we've had lots of conversations over at flake8-pyi over whether we should implement more rules that detect redundant unions (PyCQA/flake8-pyi#283, PyCQA/flake8-pyi#414, etc.). Our conclusion every time has been that without very sophisticated multifile type inference, a generalised way of detecting redundant union elements is impossible, so we should only implement rules that detect common mistakes that we've actually encountered in real code.

@sbrugman
Copy link
Contributor Author

sbrugman commented Nov 14, 2024

Hmm, how common is this antipattern in practice? I see a few ecosystem hits here, but not enough to make me think that this is something that a lot of users are doing. Has this rule been requested by any of our users?

It's a fair concern, but I would argue to give it a try and see how it turns out for users.

I've searched before implementing this rule, and it's less rare than I expected (see list below).
The rule is under preview, so we could always later decide to remove it if users do not pick it up/disable it.

For the nested unions and literals, having the rules in place can simplify the auto fixes of other rules, since we can reduce the problem to one that we've already solved.

We also should remind ourselves that the ecosystem results are published code, it could be that while developing/refactoring, such rules become more relevant. For instance saving users time when running ruff on save.

It's good to consider that is that if you expect typed Python to be more common with the amazing tooling that is coming available (type checkers, auto typing etc.), the number of users of these features also increase, and so will their need for learning new concepts such as Literal. I like to think of rules as a tool for users who are learning new concepts such as typing, and then realising when Literal[True, False] is equivalent to bool and when not could help in developing their mental model. A bit off topic, but this would be one of the cool outcomes of grouping rules related to the same language features together (#1774).

One final and exciting motivation is that these rules add to the diversity of test cases too, which can help testing red-knot!

Some more results:

@AlexWaygood
Copy link
Member

I like to think of rules as a tool for users who are learning new concepts such as typing, and then realising when Literal[True, False] is equivalent to bool and when not could help in developing their mental model. A bit off topic, but this would be one of the cool outcomes of grouping rules related to the same language features together (#1774).

Absolutely! I'm sure that this rule would help some people.

My concern is just that there's sort of an endless number of rules like this, regarding redundant union members, that you could plausibly add to Ruff. It's not clear where to draw the line, and they all add a certain amount of maintenance burden for us. When we adopt red-knot as the new backend, moreover, we should be able to consolidate them all into a single rule that is implemented with much less code and has much greater accuracy.

So I'm not against adding these rules when they address very common user issues or when we've been explicitly asked for a rule like this. If I were dead-set against it, I wouldn't have added rules like this to flake8-pyi in the first place! But I do think the bar needs to be ~reasonably high for this kind of rule.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chatted with @MichaReiser offline and he convinced me that this would be an especially helpful rule for Python beginners. I'm still a bit concerned that there'll be a lot of churn for users if we end up deprecating all these rules and replacing them with a single, generalised rule when we switch to using red-knot as the backend. But that's probably a long way off.

Just a few docs nitpicks, then I think we're good to go 👍

Comment on lines 15 to 18
/// The `bool` type has exactly two constant instances: `True` and `False`. Writing
/// `Literal[True, False]` in the context of type annotations could be replaced by a
/// bare `bool` annotations. Static type checkers such as [mypy] treat them as
/// equivalent.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Start off with a short sentence that clearly states that this is a rule concerned with readability and style rather than correctness:

Suggested change
/// The `bool` type has exactly two constant instances: `True` and `False`. Writing
/// `Literal[True, False]` in the context of type annotations could be replaced by a
/// bare `bool` annotations. Static type checkers such as [mypy] treat them as
/// equivalent.
/// `Literal[True, False]` can be replaced with `bool` in type annotations,
/// which means the same thing but is more concise and more readable.
///
/// This is because the `bool` type has exactly two constant instances:
/// `True` and `False`. Static type checkers such as [mypy] treat
/// `Literal[True, False]` as equivalent to `bool` in a type annotation.

Comment on lines 22 to 23
/// Literal[True, False]
/// Literal[True, False, "hello", "world"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Literal[True, False]
/// Literal[True, False, "hello", "world"]
/// from typing import Literal
///
/// x: Literal[True, False]
/// y: Literal[True, False, "hello", "world"]

Comment on lines 28 to 29
/// bool
/// Literal["hello", "world"] | bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// bool
/// Literal["hello", "world"] | bool
/// from typing import Literal
///
/// x: bool
/// y: Literal["hello", "world"] | bool

/// - `bool` is not strictly equivalent to `Literal[True, False]`, as `bool` is
/// a subclass of `int`, and this rule might not apply if the type annotations are used
/// in a numerical context.
/// - The `Literal` might contain comments.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// - The `Literal` might contain comments.
/// - The `Literal` slice might contain trailing-line comments which the fix would
/// remove.

Comment on lines 36 to 40
/// treat `bool` as equivalent when overloading boolean arguments with `Literal[True]`
/// and `Literal[False]`, e.g. see [#14764] and [#5421].
/// - `bool` is not strictly equivalent to `Literal[True, False]`, as `bool` is
/// a subclass of `int`, and this rule might not apply if the type annotations are used
/// in a numerical context.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// treat `bool` as equivalent when overloading boolean arguments with `Literal[True]`
/// and `Literal[False]`, e.g. see [#14764] and [#5421].
/// - `bool` is not strictly equivalent to `Literal[True, False]`, as `bool` is
/// a subclass of `int`, and this rule might not apply if the type annotations are used
/// in a numerical context.
/// treat `bool` as equivalent when overloading boolean arguments with `Literal[True]`
/// and `Literal[False]`, e.g. see [#14764] and [#5421].
/// - `bool` is not strictly equivalent to `Literal[True, False]`, as `bool` is
/// a subclass of `int`, and this rule might not apply if the type annotations are used
/// in a numerical context.

Comment on lines 38 to 40
/// - `bool` is not strictly equivalent to `Literal[True, False]`, as `bool` is
/// a subclass of `int`, and this rule might not apply if the type annotations are used
/// in a numerical context.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(arguably it's a type-checker bug if they don't understand that <object of type Literal[True]> + <object of type Literal[True]> == <object of type Literal[2]> in a numeric context. But hey.)

@charliermarsh
Copy link
Member

Applying @AlexWaygood comments, then merging.

@charliermarsh charliermarsh enabled auto-merge (squash) November 16, 2024 18:32
@charliermarsh charliermarsh merged commit 1fbed6c into astral-sh:main Nov 16, 2024
19 checks passed
dylwil3 pushed a commit to dylwil3/ruff that referenced this pull request Nov 17, 2024
## Summary

Implements `redundant-bool-literal`

## Test Plan

<!-- How was it tested? -->

`cargo test`

The ecosystem results are all correct, but for `Airflow` the rule is not
relevant due to the use of overloading (and is marked as unsafe
correctly).

---------

Co-authored-by: Charlie Marsh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants