False-positive on TypeVar bound class return value

[SalomonSmeke]

Bug Report

When using a bound TypeVar, returning a type revealed as a class (not the generic) sometimes results in a false positive.

from typing import TypeVar

class A:
    pass
class B(A):
    pass

QBT = TypeVar("QBT", bound=A)

def fn(t: QBT) -> QBT:
    if not isinstance(t, B):
        raise NotImplementedError
    return t # <- Incompatible types in "return" (actual type "B", expected type "QBT")

To Reproduce

Minimal example available here: https://mypy-play.net/?mypy=latest&python=3.10&gist=a9419e2d2627a65ae181f16224680618

On python 3.10 and latest mypy. Though I originally reproduced it on python 3.8 and mypy 0.8.

Expected Behavior

I am not 100%, but I think this should be allowed. I checked in the gitter.im and @JelleZijlstra also thought it might be a bug.

Actual Behavior

Exception regarding return type. Happens with yields also.

Your Environment

• Mypy version used: .8 and latest
• Mypy command-line flags: None
• Mypy configuration options from mypy.ini: Defaults
• Python version used: 3.8 and 3.10
• Operating system and version: macOS Big Sur / Centos7 / whatever mypy playground uses.

Workaround:

On .8 this does not work. But:

from typing import TypeVar, cast

class A:
    pass
class B(A):
    pass

QBT = TypeVar("QBT", bound=A)

def fn(t: QBT) -> QBT:
    if not isinstance(t, B):
        raise NotImplementedError
    return cast(QBT, t) # <- OK! on .8 its marked as redundant.

[erictraut]

I recently added support for this case in pyright. It took me a while to work out a good solution, so I'll share it here in case it might be of use for someone attempting to fix this in mypy.

When a symbol whose type is defined by a type variable (such as in the example above, where t is of type QBT) is narrowed using an isinstance or issubclass test, the type checker needs to remember that 1) it is now narrowed to the specified type (in the example above, B) and 2) it originated from the type variable. The second part is important to avoid a false positive error like the one reported in this example.

I solved the problem in pyright by introducing the notion of a "conditional type". A conditional type is like a normal type except that it's conditioned on one or more type variables from which it derives. In the case of constrained type variables, a type is conditioned on a particular constraint of a type variable. The example above uses a bound type variable, so there is no specific constraint associated with this condition. Pyright reports conditional types with a * after them to indicate that there is one or more type variable conditions associated with it.

def fn(t: QBT) -> QBT:
    reveal_type(t)  # Type of "t" is "QBT@fn"
    if not isinstance(t, B):
        raise NotImplementedError

    reveal_type(t)  # Type of "t" is "B*"
    return t

This same approach allows constrained type variables to be analyzed in a single pass, versus the multi-pass approach currently used in mypy.

FloatOrStr = TypeVar("FloatOrStr", float, str)

def add(v1: FloatOrStr, v2: FloatOrStr) -> FloatOrStr:
    reveal_type(v1)  # Type of "v1" is "FloatOrStr@add"
    reveal_type(v2)  # Type of "v2" is "FloatOrStr@add"

    v3 = v1 + v2
    reveal_type(v3)  # Type of "v3" is "float* | str*"

    if isinstance(v3, float):
        reveal_type(v3)  # Type of "v3" is "float*"
        return v3 + 1
    else:
        reveal_type(v3)  # Type of "v3" is "str*"
        return v3 + "1"

[LutingWang]

May be relevant to this bug. In the following code, mypy seems to ignore the type guard and requests me to return an object that is both tuple and dict.

from typing import TypeVar, Tuple, Dict

T = TypeVar('T', Tuple, Dict)

class TupleDict(tuple, dict):
    pass

def func(feat: T) -> T:
    if isinstance(feat, tuple):
        # return tuple()  # error: Incompatible return value type (got "Tuple[<nothing>, ...]", expected "Dict[Any, Any]")
        # return {}  # error: Incompatible return value type (got "Dict[<nothing>, <nothing>]", expected "Tuple[Any, ...]")
        return TupleDict()  # pass
    if isinstance(feat, dict):
        return {}

mypy 0.960 (compiled: yes)
Python 3.8.9

[shawnwall]

@LutingWang were you ever able to work around your issue in any way? I'm trying to do the same thing where I essentially am saying 'lets make up this new type that can actually be one of these 5 classes'. If I return an instance of any of those classes, mypy is complaining.

[LutingWang]

@LutingWang were you ever able to work around your issue in any way? I'm trying to do the same thing where I essentially am saying 'lets make up this new type that can actually be one of these 5 classes'. If I return an instance of any of those classes, mypy is complaining.

For now I'm using overload, similar to this post. @shawnwall

[starhel]

I've probably encountered same bug when using dataclasses. Reproduction:

from dataclasses import dataclass, is_dataclass
from typing import TypeVar, Type, Dict

T = TypeVar('T')

@dataclass
class MypyTest:
    x: int
    y: float


def make_dataclass(cls: Type[T], data: Dict) -> T:
    assert is_dataclass(cls)
    return cls(**data)


make_dataclass(MypyTest, {"x": 1, "y": 2.3})

For now I have no idea how to workaround this case, overload is not an option for me.