Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash when **unpacking kwargs typed with a TypedDict into a function typed with ParamSpec in expand_actual_type #16616

Closed
jmahlik opened this issue Dec 4, 2023 · 1 comment · Fixed by #17358

Comments

@jmahlik
Copy link

jmahlik commented Dec 4, 2023

Crash Report
This originally occurred when trying to type hint keyword args to a concurrent.Futures.ThreadPoolExecutor.submit. I was able to get it down to a more minimal reproducer. Seems that any kwargs **unpacking where the kwargs are a typed dict can trigger it.

Traceback

mypy ./typed_dict_crash.py --show-traceback

typed_dict_crash.py:37: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.8.0+dev.7c33e7c03444ae748b82163e7b4e1666dfaf94c7
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File ".venv\Scripts\mypy.exe\__main__.py", line 7, in <module>
    sys.exit(console_entry())
  File ".venv\Lib\site-packages\mypy\__main__.py", line 15, in console_entry
    main()
  File ".venv\Lib\site-packages\mypy\main.py", line 100, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File ".venv\Lib\site-packages\mypy\main.py", line 182, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File ".venv\Lib\site-packages\mypy\build.py", line 191, in build
    result = _build(
  File ".venv\Lib\site-packages\mypy\build.py", line 265, in _build
    graph = dispatch(sources, manager, stdout)
  File ".venv\Lib\site-packages\mypy\build.py", line 2943, in dispatch
    process_graph(graph, manager)
  File ".venv\Lib\site-packages\mypy\build.py", line 3341, in process_graph
    process_stale_scc(graph, scc, manager)
  File ".venv\Lib\site-packages\mypy\build.py", line 3442, in process_stale_scc
    graph[id].type_check_first_pass()
  File ".venv\Lib\site-packages\mypy\build.py", line 2311, in type_check_first_pass
    self.type_checker().check_first_pass()
  File ".venv\Lib\site-packages\mypy\checker.py", line 481, in check_first_pass
    self.accept(d)
  File ".venv\Lib\site-packages\mypy\checker.py", line 591, in accept
    stmt.accept(self)
  File ".venv\Lib\site-packages\mypy\nodes.py", line 787, in accept
    return visitor.visit_func_def(self)
  File ".venv\Lib\site-packages\mypy\checker.py", line 1000, in visit_func_def
    self._visit_func_def(defn)
  File ".venv\Lib\site-packages\mypy\checker.py", line 1004, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File ".venv\Lib\site-packages\mypy\checker.py", line 1077, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File ".venv\Lib\site-packages\mypy\checker.py", line 1293, in check_func_def
    self.accept(item.body)
  File ".venv\Lib\site-packages\mypy\checker.py", line 591, in accept
    stmt.accept(self)
  File ".venv\Lib\site-packages\mypy\nodes.py", line 1223, in accept
    return visitor.visit_block(self)
  File ".venv\Lib\site-packages\mypy\checker.py", line 2770, in visit_block
    self.accept(s)
  File ".venv\Lib\site-packages\mypy\checker.py", line 591, in accept
    stmt.accept(self)
  File ".venv\Lib\site-packages\mypy\nodes.py", line 1310, in accept
    return visitor.visit_assignment_stmt(self)
  File ".venv\Lib\site-packages\mypy\checker.py", line 2818, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File ".venv\Lib\site-packages\mypy\checker.py", line 3026, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue, type_context=type_context)
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 5727, in accept
    typ = node.accept(self)
          ^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\nodes.py", line 1904, in accept
    return visitor.visit_call_expr(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 473, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 602, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 1440, in check_call_expr_with_callee_type
    ret_type, callee_type = self.check_call(
                            ^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 1535, in check_call
    return self.check_callable_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 1720, in check_callable_call
    callee = self.infer_function_type_arguments(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\checkexpr.py", line 2045, in infer_function_type_arguments
    inferred_args, _ = infer_function_type_arguments(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\infer.py", line 56, in infer_function_type_arguments
    constraints = infer_constraints_for_callable(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\constraints.py", line 226, in infer_constraints_for_callable
    actual_type = mapper.expand_actual_type(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv\Lib\site-packages\mypy\argmap.py", line 248, in expand_actual_type
    formal_name = (set(actual_type.items.keys()) - self.kwargs_used).pop()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'pop from an empty set'
typed_dict_crash.py:37: : note: use --pdb to drop into pdb

To Reproduce

from typing import Callable, ParamSpec, TypedDict, TypeVar

_T = TypeVar("_T")
_P = ParamSpec("_P")


def submit(__fn: Callable[_P, _T], *args: _P.args, **kwargs: _P.kwargs) -> _T:
    """Similar to concurrent.futures.ThreadPoolExecutor.submit.

    https://github.com/python/typeshed/blob/0f7241844eb2692fca8b43994f0e238c50d48d46/stdlib/concurrent/futures/_base.pyi#L51

    Minimal stub to repro.
    """
    return __fn(*args, **kwargs)


def f(one: str | None = None, **kwargs: str) -> str | None:
    print(kwargs)
    return one


class D(TypedDict):
    """Keyword args to f."""

    one: str
    two: str


def func(
    typed_dict: D,
) -> None | str:
    """Pass kwargs to an underlying function via a typed dict."""
    # Originally this was a concurrent.futures.ThreadPoolExecutor.submit
    result = submit(f, **typed_dict)
    print(result)

    return result


# Both cases will crash
d = D(one="hello", two="world")
submit(f, **d)

func({"one": "hello", "two": "world"})

Your Environment

  • Mypy version used: 1.7.1 and current main at 7c33e7c03444ae748b82163e7b4e1666dfaf94c7
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.12, 3.11
  • Operating system and version: Windows 10
@jmahlik jmahlik added the crash label Dec 4, 2023
@AlexWaygood AlexWaygood added topic-typed-dict topic-paramspec PEP 612, ParamSpec, Concatenate labels Dec 4, 2023
@hauntsaninja
Copy link
Collaborator

Someone at work just ran into this, here's the repro I minimised from that:

from typing import *

P = ParamSpec("P")
R = TypeVar("R")

def run(func: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> R: ...

class Params(TypedDict):
    temperature: float

class BRTC:
    def complete(self, c: Callable[..., None], **params: Unpack[Params]) -> None:
        run(c, **params)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants