[red-knot] more precise types from chained boolean expressions

[carljm]

We currently don't eliminate some impossible types in chained boolean or comparison expressions.

from typing import Literal
class A: ...
def f(x: Literal[0, 1]):
    # Currently we reveal `Literal[0, 1] | A` in each of these cases, but we should reveal what's shown below:
    reveal_type(x or A())  # revealed: Literal[1] | A
    reveal_type(x and A())  # revealed: Literal[0] | A

This one is an existing test in comparison/non_bool_returns.md:

from __future__ import annotations

class A:
    def __lt__(self, other) -> A: ...

y = 0 < 1 < A() < 3
# Currently we reveal `bool | A` here, but it's not possible that we'd ever get `True` here, so it should be `Literal[False] | A`
reveal_type(y)  # revealed: Literal[False] | A

Both of these can be fixed by adding negative intersections with Type::AlwaysTruthy or Type::AlwaysFalsy in the appropriate cases in TypeInferenceBuilder::infer_chained_boolean_types.

[Slyces]

If I understand correctly

• We have 2 new variants Type::Truthy and Type::Falsy
• They only make sense in intersections (?)
  • A & Truthy -> subset of instances of A that evaluate to True
  • A & Falsy -> subset of instances of A that evaluate to False (0 for int/float, "" for str, ...)

I have some questions:

• Would those new variants make sense outside of intersections?
• Would we display those variants? I think I see the use - we understand that it's not "any" A - but would this ever make it to the end user?
  • If we answer no (it's purely internal to have more precise information) how do we ensure this doesn't leak?

[carljm]

Those are great questions!

Any type "makes sense" independently as long as we can describe it as some set of possible runtime values. So these types can stand alone, in principle: Type::Falsy is the set of all objects whose __bool__ method (always) returns False, and vice versa for Type::Truthy.

In practice, I expect that we would only create these types in intersections, wherever we are able to determine that a type must be truthy or falsy.

If we introduce this type, we accept (at least for now) the fact that it may leak to users, since we won't be able to simplify it in all cases. We will often have this tradeoff, where we can more fully represent our type knowledge, at the cost of displaying more complex types to users. In general, I would prefer to represent the most precise type knowledge we can be confident of, and then find ways to simplify the user type display if that proves to be necessary. For instance, we could later implement logic in the display code for intersections to just hide these types in the output, if we end up feeling that it becomes too noisy otherwise. My hope, though, is that users will find these types reasonably intuitive, and will appreciate more precise typing.

One thing to note here is that currently (since we've so far only implemented support for one form of narrowing, is not None), intersections don't occur often; implementing this issue will add another case where intersections appear, and may require adding support for intersections in more places in order to get useful test results.

[Slyces]

So, I started working on this. I have the following question: when expressing X is Truthy, do we go through the positive or the negative part of intersection?

Let's take the following example:

def f(x: int):
    if x:
        ...  # here, `x` is `Truthy` or `x` is not `Falsy`

[AlexWaygood]

Related to @Slyces's question here... do we need a Truthy variant and a Falsey variant, given that we support negative intersections? Since the set of all possible truthy objects is disjoint with the set of all possible falsey objects, str & Falsey is surely the same as str & ~Truthy (and both will simplify to Literal[""], I suppose!) So can we make do with just a Type::Truthy variant?

[carljm]

I think we could probably get away with only the Truthy variant, but I suspect it may work out better (and more efficiently) in the implementation, and be clearer in displayed types, if we have both Truthy and Falsy variants, and only use the positive side of the intersection. That is, if we try to add ~Truthy to an intersection, that turns into adding Falsy instead.

[carljm]

With the merge of #14687, this task should now be pretty trivial. We now have AlwaysFalsy and AlwaysTruthy types, we just need to add negative intersections with them in the chained boolean expressions code.