Add unbound typevar check #13166

AurelienJaquier · 2022-07-17T15:25:02Z

Description

This should fix Issue #13061 .

Mypy should now raise an error when it encounters unbound plain TypeVar.
It should not get triggered when the return type contains a TypeVar (e.g. List[T]).
It should not get triggered when a same TypeVar is present in the scope (e.g. inner function returns T, and T is an argument of outer function).

Test Plan

5 tests were added:

a plain unbound typevar triggering the new error
a typevar inner function not triggering the error
a function returning an iterable of typevars, not triggering the error
a plain bound typevar, not triggering the error
nested bound typevars, not triggering the error

We also changed the other functions triggering our error for the tests to pass.

This is our 1st contribution to Mypy. Please, guide us if there is anything we can improve in this PR.

This PR was made with @anilbey

…r/mypy into unbound

…enJaquier/mypy into add-unbound-typevar-check

ilevkivskyi

Thanks, looks good! (assuming the tests pass)

anilbey · 2022-07-17T15:49:40Z

Good news! 🎉

hauntsaninja

Could you add a test for upper bounded and constrained type vars?

I would also avoid opening PRs to projects that add type: ignore, since they likely won't be able to merge the PR until there's a new version of mypy. Besides, fixes are much preferred to adding type-ignores. If we just want to type-ignore most instances of this error, it'd be a sign that we shouldn't add it ;-)

hauntsaninja · 2022-07-17T17:44:05Z

mypy/checker.py

+        for argtype in typ.arg_types:
+            argtype.accept(arg_type_visitor)
+
+        if typ.ret_type not in arg_type_visitor.arg_types and typ.ret_type in typ.variables:


Could we micro optimise this by first checking if typ.ret_type in typ.variables and only then doing the collection of arg types?

addressed in the latest push

hauntsaninja · 2022-07-17T17:47:58Z

test-data/unit/check-typevar-unbound.test

+def f(a: Union[int, T], b: str) -> T:
+    ...
+
+def g(a: Callable[..., T], b: str) -> T:


could you add one more test case with something more nested, say List[Union[Callable[..., Tuple[T]]]]

Done with the last push.

hauntsaninja · 2022-07-17T18:06:10Z

If the typevar is upper bounded or constrained, we could actually suggest using the upper bound or the union of the constraints as the return type. This seems very common, e.g. see the pandas, spark, etc cases.

ilevkivskyi · 2022-07-17T21:32:27Z

Btw, this reminds me some people use returning a plain unbound type var with values as a way to emulate unsafe unions. But as many of you know I don't like unsafe unions, so I am fine with breaking such hacks.

ilevkivskyi · 2022-07-17T21:34:13Z

@anilbey @AurelienJaquier it would be great if you could implement some of the suggestions, but if you don't have time or want to do it in a separate PR it is also OK, please let me know.

anilbey · 2022-07-18T15:36:24Z

Thanks @hauntsaninja for the feedback.
@ilevkivskyi I just added the deeper nested tests and micro optimisation. For upper bounded and constrained types, I will need a bit of more time.

ilevkivskyi · 2022-07-18T18:20:21Z

OK, you can make those other changes in a separate PR. I will merge this when the tests pass.

ilevkivskyi · 2022-07-18T18:40:16Z

@anilbey Unfortunately one of the tests you added is failing, probably you need to add a builtins fixture that includes tuple (you can see how it is done in other tests).

…r/mypy into unbound

github-actions · 2022-07-18T22:12:30Z

Diff from mypy_primer, showing the effect of this PR on open source code:

pegen (https://github.com/we-like-parsers/pegen)
+ src/pegen/parser.py:25: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]
+ src/pegen/parser.py:45: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

pandera (https://github.com/pandera-dev/pandera)
+ pandera/engines/engine.py:184: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

paasta (https://github.com/yelp/paasta)
+ paasta_tools/utils.py:3941: error: A function returning TypeVar should receive at least one argument containing the same Typevar
+ paasta_tools/utils.py:3948: error: A function returning TypeVar should receive at least one argument containing the same Typevar

anyio (https://github.com/agronholm/anyio)
+ src/anyio/from_thread.py:122: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

vision (https://github.com/pytorch/vision)
+ torchvision/prototype/datasets/utils/_internal.py:89: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

spark (https://github.com/apache/spark)
+ python/pyspark/ml/wrapper.py:267: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/queues.py:384: error: A function returning TypeVar should receive at least one argument containing the same Typevar
+ tornado/queues.py:421: error: A function returning TypeVar should receive at least one argument containing the same Typevar

schemathesis (https://github.com/schemathesis/schemathesis)
+ src/schemathesis/auth.py: note: In member "get" of class "AuthProvider":
+ src/schemathesis/auth.py:36: error: A function returning TypeVar should receive at least one argument containing the same Typevar

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/ext/commands/commands.py:1046: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]
+ steam/ext/commands/commands.py:1077: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

core (https://github.com/home-assistant/core)
+ homeassistant/helpers/singleton.py:29: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/dependencies/data.py:321: error: A function returning TypeVar should receive at least one argument containing the same Typevar  [type-var]

anilbey · 2022-07-18T22:28:38Z

OK, you can make those other changes in a separate PR. I will merge this when the tests pass.

Alright, sounds good

Changed `Typevar` -> `TypeVar` in the error message from python#13166. I was testing the mypy 0.980 pre-release builds and noticed that the error message was using inconsistent capitalization.

Changed `Typevar` -> `TypeVar` in the error message from #13166. I was testing the mypy 0.980 pre-release builds and noticed that the error message was using inconsistent capitalization.

Also don't complain about other TypeVarLikeTypes Implements python#13166 (comment)

Also don't complain about other TypeVarLikeTypes Implements #13166 (comment)

evbo · 2024-12-22T23:03:45Z

@AurelienJaquier I was hoping to be able to do things like:

def my_func[T: Upper]() -> T:
    ...

a: MyType = my_func()

Now, to avoid cast and not rely on Any or no typing, I must do:

def my_func[T: Upper](_: Optional[T] = None) -> T:
    ...

a: MyType = my_func()

Did you consider this use-case? [T: UpperType] that's still "unbounded" to you?

Jaquier Aurélien Tristan and others added 17 commits July 17, 2022 11:16

Add simple unbound typevar check and rudimentary test

cc8d33f

Add test for unbound func returning iterables of TypeVars

cc8faff

Add appropriate error message and make the new test pass

a771393

add CollectArgTypes to get set of argument types

714445e

Merge branch 'add-unbound-typevar-check' of github.com:AurelienJaquie…

595ec8c

…r/mypy into unbound

lint fix

db501a9

fix type error

e31f967

extract check_unbound_return_typevar as method

951a54d

check if return type is instantiated in typ.variables

b9b1561

Fix some tests that are affected by new implementation

0335cc1

Merge branch 'add-unbound-typevar-check' of https://github.com/Aureli…

0cf8e93

…enJaquier/mypy into add-unbound-typevar-check

add testInnerFunctionTypeVar test

1ddf301

fix testErrorCodeNeedTypeAnnotation test

32c0c84

add TYPE_VAR error code to unbound error failure

1175d53

Merge branch 'add-unbound-typevar-check' of https://github.com/Aureli…

f1a27d1

…enJaquier/mypy into add-unbound-typevar-check

Fix the tests

402a4e3

move new tests in new test file

34dc475

ilevkivskyi approved these changes Jul 17, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

hauntsaninja reviewed Jul 17, 2022

View reviewed changes

anilbey added 2 commits July 18, 2022 17:24

add 'check-typevar-unbound.test' to testcheck

b78b926

add more nested tests for unbound type

ce9ceaf

microoptimise check_unbound_return_typevar check ret_type first

64b1834

Merge branch 'master' into add-unbound-typevar-check

0ff4b12

This comment has been minimized.

Sign in to view

anilbey added 2 commits July 18, 2022 23:43

add missing builtins fixture to failing test

4352bc3

Merge branch 'add-unbound-typevar-check' of github.com:AurelienJaquie…

abb6f1c

…r/mypy into unbound

ilevkivskyi merged commit 2c152e6 into python:master Jul 19, 2022

AurelienJaquier deleted the add-unbound-typevar-check branch July 20, 2022 06:52

intgr mentioned this pull request Sep 19, 2022

Use consistent capitalization of TypeVar #13687

Merged

ilevkivskyi mentioned this pull request Sep 25, 2022

No error in function definition that returns unbound T #13061

Closed

hauntsaninja added a commit to hauntsaninja/mypy that referenced this pull request Sep 25, 2022

Suggest using upper bound for unbound tvar

f1c41c1

Also don't complain about other TypeVarLikeTypes Implements python#13166 (comment)

hauntsaninja mentioned this pull request Sep 25, 2022

Suggest using upper bound for unbound tvar #13730

Merged

hauntsaninja added a commit that referenced this pull request Sep 26, 2022

Suggest using upper bound for unbound tvar (#13730)

6a4c2c8

Also don't complain about other TypeVarLikeTypes Implements #13166 (comment)

ilevkivskyi pushed a commit that referenced this pull request Oct 2, 2022

Suggest using upper bound for unbound tvar (#13730)

d03f201

Also don't complain about other TypeVarLikeTypes Implements #13166 (comment)

jesusbv mentioned this pull request Oct 17, 2022

Weaken type check on Result.load() return type OSInside/kiwi#2196

Merged

JukkaL mentioned this pull request Nov 19, 2022

Parameterless generic functions are allowed #2885

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add unbound typevar check #13166

Add unbound typevar check #13166

AurelienJaquier commented Jul 17, 2022

ilevkivskyi left a comment

anilbey commented Jul 17, 2022

This comment has been minimized.

hauntsaninja left a comment

hauntsaninja Jul 17, 2022

anilbey Jul 18, 2022

anilbey Jul 18, 2022

hauntsaninja Jul 17, 2022

anilbey Jul 18, 2022

hauntsaninja commented Jul 17, 2022 •

edited

Loading

ilevkivskyi commented Jul 17, 2022

ilevkivskyi commented Jul 17, 2022

anilbey commented Jul 18, 2022

ilevkivskyi commented Jul 18, 2022

ilevkivskyi commented Jul 18, 2022

This comment has been minimized.

github-actions bot commented Jul 18, 2022

anilbey commented Jul 18, 2022 •

edited

Loading

evbo commented Dec 22, 2024 •

edited

Loading

Add unbound typevar check #13166

Add unbound typevar check #13166

Conversation

AurelienJaquier commented Jul 17, 2022

Description

Test Plan

ilevkivskyi left a comment

Choose a reason for hiding this comment

anilbey commented Jul 17, 2022

This comment has been minimized.

hauntsaninja left a comment

Choose a reason for hiding this comment

hauntsaninja Jul 17, 2022

Choose a reason for hiding this comment

anilbey Jul 18, 2022

Choose a reason for hiding this comment

anilbey Jul 18, 2022

Choose a reason for hiding this comment

hauntsaninja Jul 17, 2022

Choose a reason for hiding this comment

anilbey Jul 18, 2022

Choose a reason for hiding this comment

hauntsaninja commented Jul 17, 2022 • edited Loading

ilevkivskyi commented Jul 17, 2022

ilevkivskyi commented Jul 17, 2022

anilbey commented Jul 18, 2022

ilevkivskyi commented Jul 18, 2022

ilevkivskyi commented Jul 18, 2022

This comment has been minimized.

github-actions bot commented Jul 18, 2022

anilbey commented Jul 18, 2022 • edited Loading

evbo commented Dec 22, 2024 • edited Loading

hauntsaninja commented Jul 17, 2022 •

edited

Loading

anilbey commented Jul 18, 2022 •

edited

Loading

evbo commented Dec 22, 2024 •

edited

Loading