From 786a436f2bbba8883deaa63974da6efb9815ce80 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sat, 13 May 2023 05:56:56 -0700 Subject: [PATCH 01/56] First docs work --- Doc/library/argparse.rst | 30 ++-- Doc/reference/compound_stmts.rst | 234 +++++++++++++++++++++++++++++ Doc/reference/executionmodel.rst | 19 ++- Doc/reference/lexical_analysis.rst | 16 +- Doc/reference/simple_stmts.rst | 24 +++ 5 files changed, 299 insertions(+), 24 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index 33e367f3ccda89..fbffa71d200735 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -57,20 +57,20 @@ the extracted data in a :class:`argparse.Namespace` object:: Quick Links for add_argument() ------------------------------ -====================== =========================================================== ========================================================================================================================== -Name Description Values -====================== =========================================================== ========================================================================================================================== -action_ Specify how an argument should be handled ``'store'``, ``'store_const'``, ``'store_true'``, ``'append'``, ``'append_const'``, ``'count'``, ``'help'``, ``'version'`` -choices_ Limit values to a specific set of choices ``['foo', 'bar']``, ``range(1, 10)``, or :class:`~collections.abc.Container` instance -const_ Store a constant value -default_ Default value used when an argument is not provided Defaults to ``None`` -dest_ Specify the attribute name used in the result namespace -help_ Help message for an argument -metavar_ Alternate display name for the argument as shown in help -nargs_ Number of times the argument can be used :class:`int`, ``'?'``, ``'*'``, or ``'+'`` -required_ Indicate whether an argument is required or optional ``True`` or ``False`` -type_ Automatically convert an argument to the given type :class:`int`, :class:`float`, ``argparse.FileType('w')``, or callable function -====================== =========================================================== ========================================================================================================================== +============================ =========================================================== ========================================================================================================================== +Name Description Values +============================ =========================================================== ========================================================================================================================== +action_ Specify how an argument should be handled ``'store'``, ``'store_const'``, ``'store_true'``, ``'append'``, ``'append_const'``, ``'count'``, ``'help'``, ``'version'`` +choices_ Limit values to a specific set of choices ``['foo', 'bar']``, ``range(1, 10)``, or :class:`~collections.abc.Container` instance +const_ Store a constant value +default_ Default value used when an argument is not provided Defaults to ``None`` +dest_ Specify the attribute name used in the result namespace +help_ Help message for an argument +metavar_ Alternate display name for the argument as shown in help +nargs_ Number of times the argument can be used :class:`int`, ``'?'``, ``'*'``, or ``'+'`` +required_ Indicate whether an argument is required or optional ``True`` or ``False`` +:ref:`type ` Automatically convert an argument to the given type :class:`int`, :class:`float`, ``argparse.FileType('w')``, or callable function +============================ =========================================================== ========================================================================================================================== Example @@ -1132,7 +1132,7 @@ command-line argument was not present:: Namespace(foo='1') -.. _type: +.. _argparse-type: type ^^^^ diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 9d1e5b6c596d9f..6359fbc004f0a4 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1205,6 +1205,8 @@ Function definitions A function definition defines a user-defined function object (see section :ref:`types`): +TODO(jelle): generics + .. productionlist:: python-grammar funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` @@ -1377,6 +1379,8 @@ Class definitions A class definition defines a class object (see section :ref:`types`): +TODO(jelle): generics + .. productionlist:: python-grammar classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite` inheritance: "(" [`argument_list`] ")" @@ -1589,6 +1593,236 @@ body of a coroutine function. The proposal that made coroutines a proper standalone concept in Python, and added supporting syntax. +.. _type-params: + +Type parameter lists +==================== + +.. versionadded:: 3.12 + +.. index:: + single: type parameters + +.. productionlist:: python-grammar + type_params: "[" `type_param` ("," `type_param`)* "]" + type_param: `typevar` | `typevartuple` | `paramspec` + typevar: `identifier` (":" `expression`)? + typevartuple: "*" `identifier` + paramspec: "**" `identifier` + +:ref:`Functions ` (including :ref:`coroutines `), +:ref:`classes ` and :ref:`type aliases ` may +contain a type parameter list:: + + def max[T](args: list[T]) -> T: + ... + + async def amax[T](args: list[T]) -> T: + ... + + class Bag[T]: + def __iter__(self) -> Iterator[T]: + ... + + def add(self, arg: T) -> None: + ... + + type ListOrSet[T] = list[T] | set[T] + +Semantically, this indicates that the function, class, or type alias is +generic over a type variable. This information is primarily used by static +type checkers, and at runtime, generic objects behave much like their +non-generic counterparts. + +Generic functions, classes, and type aliases have a :attr:`__type_params__` +attribute listing their type parameters. + +Type parameters come in three kinds: + +* :data:`typing.TypeVar`, introduced by a plain name (e.g., `T`). Semantically, this + stands for a single type. +* :data:`typing.TypeVarTuple`, introduced by a name prefixed with a single + asterisk (e.g., `*Ts`). Semantically, this stands for a tuple of any + number of types. +* :data:`typing.ParamSpec`, introduced by a name prefixed with two asterisks + (e.g., `**P`). Semantically, this stands for the parameters of a callable. + +:data:`typing.TypeVar` declarations can define *bounds* and *constraints* with +a colon (`:`) followed by an expression. A single expression after the colon +indicates a bound (e.g. `T: int`). The expression should be a type (though this +is not enforced at runtime, only by static type checkers) and semantically, this means +that the :data:`!typing.TypeVar` can only represent types that are a subtype of +this bound. A parenthesized tuple of expressions after the colon indicates a +set of constraints (e.g. `T: (str, bytes)`). Each member of the tuple should be a +type (again, this is not enforced at runtime). Constrained type variables can only +take on one of the types in the list of constraints. + +For :data:`!typing.TypeVar`\ s declared using the type parameter list syntax, +the bound and constraints are not evaluated when the generic object is created, +but only when the value is explicitly accessed through the attributes ``__bound__`` +and ``__constraints__``. To accomplish this, the bounds or constraints are +evaluated in a separate :ref:`type scope `. + +:data:`typing.TypeVarTuple`\ s and :data:`typing.ParamSpec`\ s cannot have bounds +or constraints. + +The following example indicates the full set of allowed type parameter declarations:: + + def overly_generic[ + SimpleTypeVar, + TypeVarWithBound: int, + TypeVarWithConstraints: (str, bytes), + *SimpleTypeVarTuple, + **SimpleParamSpec, + ]( + a: SimpleTypeVar, + b: TypeVarWithBound, + c: Callable[SimpleParamSpec, TypeVarWithConstraints], + *d: SimpleTypeVarTuple, + ): ... + +.. _type-scopes: + +Type scopes +----------- + +Type parameter lists and :keyword:`type` statements introduce *type scopes*, +which behave mostly like function scopes, but with some exceptions. Below, +the semantics of type parameter lists are explained using the pseudo-keyword +`def'` to indicate a type scope. + +Type scopes are used in the following contexts: + +* Type parameter lists for generic type aliases +* Type parameter lists for generic functions. The function's annotations are + executed within the type scope, but its defaults and decorators are not. +* Type parameter lists for generic classes. The class's base classes and + keyword arguments are executed within the type scope, but its decorators are not. +* The bounds and constraints for type variables. +* The value of type aliases. + +Type scopes differ from function scopes in the following ways: + +* If a type scope is immediately within a class scope, or within another + type scope that is immediately within a class scope, names defined in the + class scope can be accessed from within the type scope. (By contrast, regular + functions defined within classes cannot access names defined in the class scope.) +* Expressions in type scopes cannot contain :keyword:`yield`, ``yield from``, + :keyword:`await`, or :token:`:= <~python-grammar:expression>` expressions. (These + expressions are allowed in other scopes contained within the type scope.) +* Names defined in type scopes cannot be rebound with :keyword:`nonlocal` + statements in inner scopes. This includes only type parameters, as no other + syntactic elements that can appear within type scopes can introduce new names. +* While type scopes internally have a name, that name is not reflected in the + :attr:`__qualname__` of objects defined within the scope. Instead, the :attr:`!__qualname__` + of such objects is as if the object was defined in the enclosing scope. + +.. _generic-functions: + +Generic functions +----------------- + +Generic functions are declared as follows:: + + def func[T](arg: T): ... + +This syntax is equivalent to:: + + def' TYPE_PARAMS_OF_func(): + T = typing.TypeVar("T") + def func(arg: T): ... + func.__type_params__ = (T,) + return func + func = TYPE_PARAMS_OF_func() + +Here `def'` indicates a :ref:`type scope `. (Two other +liberties are taken in the translation: the ``__type_params__`` +attribute of generic functions is not writable from Python code; +and the syntax does not go through attribute access on the :mod:`typing` +module, but creates an instance of :data:`typing.TypeVar` directly.) + +The annotations of generic functions are defined within the type scope +used for declaring the type parameters, but the function's defaults and +decorators are not. + +The following example illustrates the scoping rules for these cases, +as well as for additional flavors of type parameters:: + + @decorator + def func[T: int, *Ts, **P](*args: *Ts, arg: Callable[P, T] = some_default): + ... + +This is equivalent to:: + + DEFAULT_OF_arg = some_default + + def' TYPE_PARAMS_OF_func(): + + def' BOUND_OF_T(): + return int + T = typing.TypeVar("T", evaluate_bound=BOUND_OF_T) + + Ts = typing.TypeVarTuple("Ts") + P = typing.ParamSpec("P") + + def func(*args: *Ts, arg: Callable[P, T] = DEFAULT_OF_arg): + ... + + func.__type_params__ = (T, Ts, P) + return func + func = decorator(TYPE_PARAMS_OF_func()) + +The Python constructor of :data:`typing.TypeVar` does not take an +``evaluate_bound`` argument. It is used in the translation to indicate +that the bound is lazily evaluated. + +.. _generic-classes: + +Generic classes +--------------- + +Generic classes are declared as follows:: + + class Bag[T]: ... + +This syntax is equivalent to:: + + def' TYPE_PARAMS_OF_Bag(): + T = typing.TypeVar("T") + class Bag(typing.Generic[T]): + __type_params__ = (T,) + ... + return Bag + Bag = TYPE_PARAMS_OF_Bag() + +Here again `def'` indicates a :ref:`type scope `. + +Generic classes implicitly inherit from :data:`typing.Generic`. +The base classes and keyword arguments of generic classes are +executed within the type scope for the type parameters, +and decorators are executed outside that scope. This is illustrated +by this example:: + + @decorator + class Bag(Base[T], arg=T): ... + +This is equivalent to:: + + def' TYPE_PARAMS_OF_Bag(): + T = typing.TypeVar("T") + class Bag(Base[T], typing.Generic[T], arg=T): + __type_params__ = (T,) + ... + return Bag + Bag = decorator(TYPE_PARAMS_OF_Bag()) + +.. _generic-type-aliases: + +Generic type aliases +-------------------- + +TODO(jelle) + .. rubric:: Footnotes diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 8917243999d399..5c55ca2482ed89 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -71,6 +71,8 @@ The following constructs bind names: + in a capture pattern in structural pattern matching * :keyword:`import` statements. +* :keyword:`type` statements. +* :ref:`type parameter lists `. The :keyword:`!import` statement of the form ``from ... import *`` binds all names defined in the imported module, except those beginning with an underscore. @@ -149,7 +151,8 @@ a global statement, the free variable is treated as a global. The :keyword:`nonlocal` statement causes corresponding names to refer to previously bound variables in the nearest enclosing function scope. :exc:`SyntaxError` is raised at compile time if the given name does not -exist in any enclosing function scope. +exist in any enclosing function scope. :ref:`Type parameters ` +cannot be rebound with the :keyword:`!nonlocal` statement. .. index:: pair: module; __main__ @@ -163,14 +166,24 @@ These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the -class block; it does not extend to the code blocks of methods -- this includes +class block; it does not extend to the code blocks of methods. This includes comprehensions and generator expressions since they are implemented using a -function scope. This means that the following will fail:: +function scope, but it does not include :ref:`type scopes `, +which have access to their enclosing class scopes. +This means that the following will fail:: class A: a = 42 b = list(a + i for i in range(10)) +However, the following will succeed:: + + class A: + type Alias = Nested + class Nested: ... + + print(A.Alias.__value__) # + .. _restrict_exec: Builtins and restricted execution diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 8adb4b740825d0..c464fb83941a1f 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -361,15 +361,19 @@ Soft Keywords .. versionadded:: 3.10 Some identifiers are only reserved under specific contexts. These are known as -*soft keywords*. The identifiers ``match``, ``case`` and ``_`` can -syntactically act as keywords in contexts related to the pattern matching -statement, but this distinction is done at the parser level, not when -tokenizing. +*soft keywords*. The identifiers ``match``, ``case``, ``type`` and ``_`` can +syntactically act as keywords in certain contexts, +but this distinction is done at the parser level, not when tokenizing. -As soft keywords, their use with pattern matching is possible while still -preserving compatibility with existing code that uses ``match``, ``case`` and ``_`` as +As soft keywords, their use in the grammar is possible while still +preserving compatibility with existing code that uses these names as identifier names. +``match``, ``case``, and ``_`` are used in the :keyword:`match` statement. +``type`` is used in the :keyword:`type` statement. + +.. versionchanged:: 3.12 + ``type`` is now a soft keyword. .. index:: single: _, identifiers diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index f7a8b44d195417..9db35baab458b6 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -28,6 +28,7 @@ simple statements is: : | `future_stmt` : | `global_stmt` : | `nonlocal_stmt` + : | `type_stmt` .. _exprstmts: @@ -1012,3 +1013,26 @@ pre-existing bindings in the local scope. :pep:`3104` - Access to Names in Outer Scopes The specification for the :keyword:`nonlocal` statement. + +.. _type: + +The :keyword:`!type` statement +============================== + +.. index:: pair: statement; type + +.. productionlist:: python-grammar + type_stmt: 'type' `identifier` [`type_params`] "=" `expression` + +The :keyword:`!type` statement introduces a type alias. + +TODO(jelle): Link to typing.TypeAliasType. Link to section on type params. Link to section on lazy eval of RHS. + +:keyword:`!type` is a :ref:`soft keyword `. + +.. versionadded:: 3.12 + +.. seealso:: + + :pep:`695` - Type Parameter Syntax + Introduced the `type` statement and syntax for generic classes and functions. From c1f266e919e04eb43ab281da3134972dca37efeb Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 15 May 2023 06:50:12 -0700 Subject: [PATCH 02/56] More words --- Doc/reference/compound_stmts.rst | 77 +++++++++++++------------------- Doc/reference/executionmodel.rst | 43 +++++++++++++++++- 2 files changed, 73 insertions(+), 47 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 6359fbc004f0a4..9f7413405c72a2 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1634,6 +1634,16 @@ generic over a type variable. This information is primarily used by static type checkers, and at runtime, generic objects behave much like their non-generic counterparts. +Type parameters are declared in square brackets (``[]``) immediately +after the name of the function, class, or type alias. The type parameters +are accessible within the scope of the generic object, but not elsewhere. +Thus, after a declaration ``def func[T](): ...``, the name ``T`` is not in +the module scope. Below, the semantics of generic objects are described +with more precision. The scope of type parameters is modeled with a special +function (technically, an :ref:`annotation scope `, +represented with the pseudo-keyword ``def'``) that +wraps the creation of the generic object. + Generic functions, classes, and type aliases have a :attr:`__type_params__` attribute listing their type parameters. @@ -1681,42 +1691,6 @@ The following example indicates the full set of allowed type parameter declarati *d: SimpleTypeVarTuple, ): ... -.. _type-scopes: - -Type scopes ------------ - -Type parameter lists and :keyword:`type` statements introduce *type scopes*, -which behave mostly like function scopes, but with some exceptions. Below, -the semantics of type parameter lists are explained using the pseudo-keyword -`def'` to indicate a type scope. - -Type scopes are used in the following contexts: - -* Type parameter lists for generic type aliases -* Type parameter lists for generic functions. The function's annotations are - executed within the type scope, but its defaults and decorators are not. -* Type parameter lists for generic classes. The class's base classes and - keyword arguments are executed within the type scope, but its decorators are not. -* The bounds and constraints for type variables. -* The value of type aliases. - -Type scopes differ from function scopes in the following ways: - -* If a type scope is immediately within a class scope, or within another - type scope that is immediately within a class scope, names defined in the - class scope can be accessed from within the type scope. (By contrast, regular - functions defined within classes cannot access names defined in the class scope.) -* Expressions in type scopes cannot contain :keyword:`yield`, ``yield from``, - :keyword:`await`, or :token:`:= <~python-grammar:expression>` expressions. (These - expressions are allowed in other scopes contained within the type scope.) -* Names defined in type scopes cannot be rebound with :keyword:`nonlocal` - statements in inner scopes. This includes only type parameters, as no other - syntactic elements that can appear within type scopes can introduce new names. -* While type scopes internally have a name, that name is not reflected in the - :attr:`__qualname__` of objects defined within the scope. Instead, the :attr:`!__qualname__` - of such objects is as if the object was defined in the enclosing scope. - .. _generic-functions: Generic functions @@ -1735,13 +1709,13 @@ This syntax is equivalent to:: return func func = TYPE_PARAMS_OF_func() -Here `def'` indicates a :ref:`type scope `. (Two other +Here ``def'`` indicates an :ref:`annotation scope `. (Two other liberties are taken in the translation: the ``__type_params__`` attribute of generic functions is not writable from Python code; and the syntax does not go through attribute access on the :mod:`typing` module, but creates an instance of :data:`typing.TypeVar` directly.) -The annotations of generic functions are defined within the type scope +The annotations of generic functions are defined within the annotation scope used for declaring the type parameters, but the function's defaults and decorators are not. @@ -1752,7 +1726,7 @@ as well as for additional flavors of type parameters:: def func[T: int, *Ts, **P](*args: *Ts, arg: Callable[P, T] = some_default): ... -This is equivalent to:: +Except for the lazy evaluation of the ``TypeVar`` bound, this is equivalent to:: DEFAULT_OF_arg = some_default @@ -1760,7 +1734,8 @@ This is equivalent to:: def' BOUND_OF_T(): return int - T = typing.TypeVar("T", evaluate_bound=BOUND_OF_T) + # In reality, BOUND_OF_T() is evaluated only on demand. + T = typing.TypeVar("T", bound=BOUND_OF_T()) Ts = typing.TypeVarTuple("Ts") P = typing.ParamSpec("P") @@ -1772,10 +1747,6 @@ This is equivalent to:: return func func = decorator(TYPE_PARAMS_OF_func()) -The Python constructor of :data:`typing.TypeVar` does not take an -``evaluate_bound`` argument. It is used in the translation to indicate -that the bound is lazily evaluated. - .. _generic-classes: Generic classes @@ -1795,7 +1766,7 @@ This syntax is equivalent to:: return Bag Bag = TYPE_PARAMS_OF_Bag() -Here again `def'` indicates a :ref:`type scope `. +Here again ``def'`` indicates an :ref:`annotation scope `. Generic classes implicitly inherit from :data:`typing.Generic`. The base classes and keyword arguments of generic classes are @@ -1821,7 +1792,21 @@ This is equivalent to:: Generic type aliases -------------------- -TODO(jelle) +The :keyword:`type` statement can also be used to create a generic type alias: + + type ListOrSet[T] = list[T] | set[T] + +Except for the lazy evaluation of the value, this is equivalent to: + + def' TYPE_PARAMS_OF_ListOrSet(): + T = typing.TypeVar("T") + + def' VALUE_OF_ListOrSet(): + return list[T] | set[T] + # In reality, the value is lazily evaluated + return typing.TypeAliasType("ListOrSet", VALUE_OF_ListOrSet(), type_params=(T,)) + ListOrSet = TYPE_PARAMS_OF_ListOrSet() + .. rubric:: Footnotes diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 5c55ca2482ed89..863d59214fe28d 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -168,7 +168,7 @@ The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods. This includes comprehensions and generator expressions since they are implemented using a -function scope, but it does not include :ref:`type scopes `, +function scope, but it does not include :ref:`annotation scopes `, which have access to their enclosing class scopes. This means that the following will fail:: @@ -184,6 +184,47 @@ However, the following will succeed:: print(A.Alias.__value__) # +.. _annotation-scopes: + +Annotation scopes +----------------- + +:ref:`Type parameter lists `` and :keyword:`type` statements +introduce *annotation scopes*, which behave mostly like function scopes, +but with some exceptions discussed below. :term:`Annotations ` +currently do not use annotation scopes, but they are expected to use +annotation scopes in Python 3.13 when :pep:`649` is implemented. + +Annotation scopes are used in the following contexts: + +* Type parameter lists for generic type aliases +* Type parameter lists for generic functions. The function's annotations are + executed within the type scope, but its defaults and decorators are not. +* Type parameter lists for generic classes. The class's base classes and + keyword arguments are executed within the type scope, but its decorators are not. +* The bounds and constraints for type variables. +* The value of type aliases. + +Annotation scopes differ from function scopes in the following ways: + +* If an annotation scope is immediately within a class scope, or within another + annotation scope that is immediately within a class scope, names defined in the + class scope can be accessed from within the annotation scope, as if the code in the + annotation scope was executing directly within the class scope. (By contrast, regular + functions defined within classes cannot access names defined in the class scope.) +* Expressions in annotation scopes cannot contain :keyword:`yield`, ``yield from ``, + :keyword:`await`, or :token:`:= <~python-grammar:expression>` expressions. (These + expressions are allowed in other scopes contained within the type scope.) +* Names defined in annotation scopes cannot be rebound with :keyword:`nonlocal` + statements in inner scopes. This includes only type parameters, as no other + syntactic elements that can appear within type scopes can introduce new names. +* While annotation scopes internally have a name, that name is not reflected in the + :attr:`__qualname__` of objects defined within the scope. Instead, the :attr:`!__qualname__` + of such objects is as if the object was defined in the enclosing scope. + +.. versionadded:: 3.12 + Annotation scopes were introduced in Python 3.12 as part of :pep:`695`. + .. _restrict_exec: Builtins and restricted execution From e3dfb2750f9317b224d2fc74b07a18a778f64be4 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 21:21:20 -0700 Subject: [PATCH 03/56] ast changes, fix build --- Doc/library/ast.rst | 102 ++++++++++++++++++++++++++++++- Doc/reference/compound_stmts.rst | 8 +-- Doc/reference/executionmodel.rst | 2 +- 3 files changed, 104 insertions(+), 8 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index eb6a973cac62d2..cc94b2755567cb 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -917,6 +917,25 @@ Statements type_ignores=[]) +.. class:: TypeAlias(name, typeparams, value) + + A type alias created through the :keyword:`type` statement. ``name`` + is the name of the alias, ``typeparams`` is a list of + :ref:`type parameters `, and ``value`` is the value of the + type alias. + + .. doctest:: + + >>> print(ast.dump(ast.parse('type Alias = int'), indent=4)) + Module( + body=[ + TypeAlias( + name=Name(id='Alias', ctx=Store()), + typeparams=[], + value=Name(id='int', ctx=Load()))], + type_ignores=[]) + + Other statements which are only applicable inside functions or loops are described in other sections. @@ -1644,15 +1663,93 @@ Pattern matching value=Constant(value=Ellipsis))])])], type_ignores=[]) +.. _ast-type-params: + +Type parameters +^^^^^^^^^^^^^^^ + +:ref:`Type parameters ` can exist on classes, functions, and type +aliases. + +.. class:: TypeVar(name, bound) + + A :class:`~typing.TypeVar`. ``name`` is the name of the type variable, and + ``bound`` is the bound or constraints, if any. If the value is a :class:`Tuple`, + it represents constraints; otherwise it represents the bound. + + .. doctest:: + + >>> print(ast.dump(ast.parse("type Alias[T: int] = list[T]"), indent=4)) + Module( + body=[ + TypeAlias( + name=Name(id='Alias', ctx=Store()), + typeparams=[ + TypeVar( + name='T', + bound=Name(id='int', ctx=Load()))], + value=Subscript( + value=Name(id='list', ctx=Load()), + slice=Name(id='T', ctx=Load()), + ctx=Load()))], + type_ignores=[]) + +.. class:: ParamSpec(name) + + A :class:`~typing.ParamSpec`. ``name`` is the name of the parameter specification. + + .. doctest:: + + >>> print(ast.dump(ast.parse("type Alias[**P] = Callable[P, int]"), indent=4)) + Module( + body=[ + TypeAlias( + name=Name(id='Alias', ctx=Store()), + typeparams=[ + ParamSpec(name='P')], + value=Subscript( + value=Name(id='Callable', ctx=Load()), + slice=Tuple( + elts=[ + Name(id='P', ctx=Load()), + Name(id='int', ctx=Load())], + ctx=Load()), + ctx=Load()))], + type_ignores=[]) + +.. class:: TypeVarTuple(name) + + A :class:`~typing.TypeVarTuple`. ``name`` is the name of the type variable tuple. + + .. doctest:: + + >>> print(ast.dump(ast.parse("type Alias[*Ts] = tuple[*Ts]"), indent=4)) + Module( + body=[ + TypeAlias( + name=Name(id='Alias', ctx=Store()), + typeparams=[ + TypeVarTuple(name='Ts')], + value=Subscript( + value=Name(id='tuple', ctx=Load()), + slice=Tuple( + elts=[ + Starred( + value=Name(id='Ts', ctx=Load()), + ctx=Load())], + ctx=Load()), + ctx=Load()))], + type_ignores=[]) Function and class definitions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. class:: FunctionDef(name, args, body, decorator_list, returns, type_comment) +.. class:: FunctionDef(name, typeparams, args, body, decorator_list, returns, type_comment) A function definition. * ``name`` is a raw string of the function name. + * ``typeparams`` is a list of :ref:`type parameters `. * ``args`` is an :class:`arguments` node. * ``body`` is the list of nodes inside the function. * ``decorator_list`` is the list of decorators to be applied, stored outermost @@ -1820,11 +1917,12 @@ Function and class definitions type_ignores=[]) -.. class:: ClassDef(name, bases, keywords, starargs, kwargs, body, decorator_list) +.. class:: ClassDef(name, typeparams, bases, keywords, starargs, kwargs, body, decorator_list) A class definition. * ``name`` is a raw string for the class name + * ``typeparams`` is a list of :ref:`type parameters `. * ``bases`` is a list of nodes for explicitly specified base classes. * ``keywords`` is a list of :class:`keyword` nodes, principally for 'metaclass'. Other keywords will be passed to the metaclass, as per `PEP-3115 diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 9f7413405c72a2..42d7c1151e6b18 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1671,7 +1671,7 @@ For :data:`!typing.TypeVar`\ s declared using the type parameter list syntax, the bound and constraints are not evaluated when the generic object is created, but only when the value is explicitly accessed through the attributes ``__bound__`` and ``__constraints__``. To accomplish this, the bounds or constraints are -evaluated in a separate :ref:`type scope `. +evaluated in a separate :ref:`annotation scope `. :data:`typing.TypeVarTuple`\ s and :data:`typing.ParamSpec`\ s cannot have bounds or constraints. @@ -1792,11 +1792,11 @@ This is equivalent to:: Generic type aliases -------------------- -The :keyword:`type` statement can also be used to create a generic type alias: +The :keyword:`type` statement can also be used to create a generic type alias:: type ListOrSet[T] = list[T] | set[T] -Except for the lazy evaluation of the value, this is equivalent to: +Except for the lazy evaluation of the value, this is equivalent to:: def' TYPE_PARAMS_OF_ListOrSet(): T = typing.TypeVar("T") @@ -1807,8 +1807,6 @@ Except for the lazy evaluation of the value, this is equivalent to: return typing.TypeAliasType("ListOrSet", VALUE_OF_ListOrSet(), type_params=(T,)) ListOrSet = TYPE_PARAMS_OF_ListOrSet() - - .. rubric:: Footnotes .. [#] The exception is propagated to the invocation stack unless diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 863d59214fe28d..a37fd8281e2c30 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -189,7 +189,7 @@ However, the following will succeed:: Annotation scopes ----------------- -:ref:`Type parameter lists `` and :keyword:`type` statements +:ref:`Type parameter lists ` and :keyword:`type` statements introduce *annotation scopes*, which behave mostly like function scopes, but with some exceptions discussed below. :term:`Annotations ` currently do not use annotation scopes, but they are expected to use From f5b1d86aa6115c25b37ce4180c507a6fecd1ef24 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 21:25:38 -0700 Subject: [PATCH 04/56] the big PEP list --- Doc/library/typing.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index c300c4257f0e81..c5cba8fe2935c3 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -101,6 +101,8 @@ annotations. These include: * :pep:`692`: Using ``TypedDict`` for more precise ``**kwargs`` typing *Introducing* a new way of typing ``**kwargs`` with :data:`Unpack` and :data:`TypedDict` +* :pep:`695`: Type Parameter Syntax + *Introducing* syntax for creating generic functions, classes, and type aliases. * :pep:`698`: Adding an override decorator to typing *Introducing* the :func:`@override` decorator From 579863196362a1ebfbe43b8423adc1befc58450a Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 21:48:18 -0700 Subject: [PATCH 05/56] dis --- Doc/library/dis.rst | 73 +++++++++++++++++++++++++++++++++---------- Doc/whatsnew/3.12.rst | 9 ++++-- 2 files changed, 63 insertions(+), 19 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 248743b8fa0a87..2a1f1704827b2e 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -926,6 +926,27 @@ iterations of the loop. .. opcode:: LOAD_NAME (namei) Pushes the value associated with ``co_names[namei]`` onto the stack. + The name is looked up within the locals, then the globals, then the builtins. + + +.. opcode:: LOAD_LOCALS + + Pushes a reference to the locals dictionary onto the stack. This is used + to prepare namespace dictionaries for :opcode:`LOAD_FROM_DICT_OR_DEREF` + and :opcode:`LOAD_FROM_DICT_OR_GLOBALS`. + + .. versionadded:: 3.12 + + +.. opcode:: LOAD_FROM_DICT_OR_GLOBALS (i) + + Pops a mapping off the stack and looks up the value for ``co_names[namei]``. + If the name is not found there, look it up in the globals and then the builtins, + similar to :opcode:`LOAD_GLOBAL`. + This is used for loading global variables in + :ref:`annotation scopes ` within class bodies. + + .. versionadded:: 3.12 .. opcode:: BUILD_TUPLE (count) @@ -1243,16 +1264,17 @@ iterations of the loop. ``i`` is no longer offset by the length of ``co_varnames``. -.. opcode:: LOAD_CLASSDEREF (i) +.. opcode:: LOAD_FROM_DICT_OR_DEREF (i) - Much like :opcode:`LOAD_DEREF` but first checks the locals dictionary before - consulting the cell. This is used for loading free variables in class - bodies. - - .. versionadded:: 3.4 + Pops a mapping off the stack and looks up the name associated with + slot ``i`` of the "fast locals" storage in this mapping. + If the name is not found there, load it from the cell contained in + slot ``i``, similar to :opcode:`LOAD_DEREF`. This is used for loading + free variables in class bodies and in + :ref:`annotation scopes ` within class bodies. - .. versionchanged:: 3.11 - ``i`` is no longer offset by the length of ``co_varnames``. + .. versionadded:: 3.12 + Replaces the previous instruction ``LOAD_CLASSDEREF``. .. opcode:: STORE_DEREF (i) @@ -1504,13 +1526,23 @@ iterations of the loop. The operand determines which intrinsic function is called: - * ``0`` Not valid - * ``1`` Prints the argument to standard out. Used in the REPL. - * ``2`` Performs ``import *`` for the named module. - * ``3`` Extracts the return value from a ``StopIteration`` exception. - * ``4`` Wraps an aync generator value - * ``5`` Performs the unary ``+`` operation - * ``6`` Converts a list to a tuple + * ``INTRINSIC_1_INVALID`` Not valid + * ``INTRINSIC_PRINT`` Prints the argument to standard out. + Used in the REPL. + * ``INTRINSIC_IMPORT_STAR`` Performs ``import *`` for the named module. + * ``INTRINSIC_STOPITERATION_ERROR`` Extracts the return value from + a ``StopIteration`` exception. + * ``INTRINSIC_ASYNC_GEN_WRAP`` Wraps an aync generator value + * ``INTRINSIC_UNARY_POSITIVE`` Performs the unary ``+`` operation + * ``INTRINSIC_LIST_TO_TUPLE`` Converts a list to a tuple + * ``INTRINSIC_TYPEVAR`` Creates a :class:`typing.TypeVar` + * ``INTRINSIC_PARAMSPEC`` Creates a :class:`typing.ParamSpec` + * ``INTRINSIC_TYPEVARTUPLE`` Creates a :class:`typing.TypeVarTuple` + * ``INTRINSIC_SUBSCRIPT_GENERIC`` Returns :class:`typing.Generic` + subscripted with the value on top of the stack. + * ``INTRINSIC_TYPEALIAS`` Creates a :class:`typing.TypeAliasType`; + used in the :keyword:`type` statement. The argument is a tuple of + the type alias's name, type parameters, and value. .. versionadded:: 3.12 @@ -1522,8 +1554,15 @@ iterations of the loop. The operand determines which intrinsic function is called: - * ``0`` Not valid - * ``1`` Calculates the :exc:`ExceptionGroup` to raise from a ``try-except*``. + * ``INTRINSIC_2_INVALID`` Not valid + * ``INTRINSIC_PREP_RERAISE_STAR`` Calculates the :exc:`ExceptionGroup` + to raise from a ``try-except*``. + * ``INTRINSIC_TYPEVAR_WITH_BOUND`` Creates a :class:`typing.TypeVar` + with a bound. + * ``INTRINSIC_TYPEVAR_WITH_CONSTRAINTS`` Creates a :class:`typing.TypeVar` + with constraints. + * ``INTRINSIC_SET_FUNCTION_TYPE_PARAMS`` Sets the ``__type_params__`` + attribute of a function. .. versionadded:: 3.12 diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 25f0a4c3ca26da..aeb534afb0ceb6 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -680,14 +680,19 @@ Optimizations CPython bytecode changes ======================== -* Removed the :opcode:`LOAD_METHOD` instruction. It has been merged into +* Remove the :opcode:`LOAD_METHOD` instruction. It has been merged into :opcode:`LOAD_ATTR`. :opcode:`LOAD_ATTR` will now behave like the old :opcode:`LOAD_METHOD` instruction if the low bit of its oparg is set. (Contributed by Ken Jin in :gh:`93429`.) -* Removed the :opcode:`!JUMP_IF_FALSE_OR_POP` and :opcode:`!JUMP_IF_TRUE_OR_POP` +* Remove the :opcode:`!JUMP_IF_FALSE_OR_POP` and :opcode:`!JUMP_IF_TRUE_OR_POP` instructions. (Contributed by Irit Katriel in :gh:`102859`.) +* Add the :opcode:`LOAD_FROM_DICT_OR_DEREF`, :opcode:`LOAD_FROM_DICT_OR_GLOBALS`, + and :opcode:`LOAD_LOCALS` opcodes as part of the implementation of :pep:`695`. + Remove the :opcode:`!LOAD_CLASSDEREF` opcode, which can be replaced with + :opcode:`LOAD_LOCALS` plus :opcode:`LOAD_FROM_DICT_OR_DEREF`. (Contributed + by Jelle Zijlstra in :gh:`103764`.) Demos and Tools =============== From 0f76679e9f623e3d192df4d76ca0d6435d6f3705 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 21:51:31 -0700 Subject: [PATCH 06/56] default roles --- Doc/reference/compound_stmts.rst | 12 ++++++------ Doc/reference/simple_stmts.rst | 3 ++- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 42d7c1151e6b18..47a2ee097c2dc7 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1649,21 +1649,21 @@ attribute listing their type parameters. Type parameters come in three kinds: -* :data:`typing.TypeVar`, introduced by a plain name (e.g., `T`). Semantically, this +* :data:`typing.TypeVar`, introduced by a plain name (e.g., ``T``). Semantically, this stands for a single type. * :data:`typing.TypeVarTuple`, introduced by a name prefixed with a single - asterisk (e.g., `*Ts`). Semantically, this stands for a tuple of any + asterisk (e.g., ``*Ts``). Semantically, this stands for a tuple of any number of types. * :data:`typing.ParamSpec`, introduced by a name prefixed with two asterisks - (e.g., `**P`). Semantically, this stands for the parameters of a callable. + (e.g., ``**P``). Semantically, this stands for the parameters of a callable. :data:`typing.TypeVar` declarations can define *bounds* and *constraints* with -a colon (`:`) followed by an expression. A single expression after the colon -indicates a bound (e.g. `T: int`). The expression should be a type (though this +a colon (``:``) followed by an expression. A single expression after the colon +indicates a bound (e.g. ``T: int``). The expression should be a type (though this is not enforced at runtime, only by static type checkers) and semantically, this means that the :data:`!typing.TypeVar` can only represent types that are a subtype of this bound. A parenthesized tuple of expressions after the colon indicates a -set of constraints (e.g. `T: (str, bytes)`). Each member of the tuple should be a +set of constraints (e.g. ``T: (str, bytes)``). Each member of the tuple should be a type (again, this is not enforced at runtime). Constrained type variables can only take on one of the types in the list of constraints. diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 9db35baab458b6..228ea795548903 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -1035,4 +1035,5 @@ TODO(jelle): Link to typing.TypeAliasType. Link to section on type params. Link .. seealso:: :pep:`695` - Type Parameter Syntax - Introduced the `type` statement and syntax for generic classes and functions. + Introduced the :keyword:`!type` statement and syntax for + generic classes and functions. From cb7602143edbff9120409c13a3d58486698b33a4 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:03:46 -0700 Subject: [PATCH 07/56] TODOs in compound_stmts --- Doc/reference/compound_stmts.rst | 41 +++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 47a2ee097c2dc7..b36fcc605eec6f 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1205,10 +1205,8 @@ Function definitions A function definition defines a user-defined function object (see section :ref:`types`): -TODO(jelle): generics - .. productionlist:: python-grammar - funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" + funcdef: [`decorators`] "def" `funcname` [`type_params`] "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` decorators: `decorator`+ decorator: "@" `assignment_expression` NEWLINE @@ -1258,6 +1256,15 @@ except that the original function is not temporarily bound to the name ``func``. :token:`~python-grammar:assignment_expression`. Previously, the grammar was much more restrictive; see :pep:`614` for details. +A list of :ref:`type parameters ` may be given in square brackets +between the function's name and the opening parenthesis for its parameter list. +This indicates to static type checkers that the function is generic. At runtime, +the type parameters can be retrieved from the function's ``__type_params__`` +attribute. See :ref:`generic-functions` for more. + +.. versionadded:: 3.12 + Type parameters are new in version 3.12. + .. index:: triple: default; parameter; value single: argument; function definition @@ -1379,10 +1386,8 @@ Class definitions A class definition defines a class object (see section :ref:`types`): -TODO(jelle): generics - .. productionlist:: python-grammar - classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite` + classdef: [`decorators`] "class" `classname` [`type_params`] [`inheritance`] ":" `suite` inheritance: "(" [`argument_list`] ")" classname: `identifier` @@ -1438,6 +1443,15 @@ decorators. The result is then bound to the class name. :token:`~python-grammar:assignment_expression`. Previously, the grammar was much more restrictive; see :pep:`614` for details. +A list of :ref:`type parameters ` may be given in square brackets +immediately after the class's name. +This indicates to static type checkers that the class is generic. At runtime, +the type parameters can be retrieved from the class's ``__type_params__`` +attribute. See :ref:`generic-classes` for more. + +.. versionadded:: 3.12 + Type parameters are new in version 3.12. + **Programmer's note:** Variables defined in the class definition are class attributes; they are shared by instances. Instance attributes can be set in a method with ``self.name = value``. Both class and instance attributes are @@ -1709,13 +1723,12 @@ This syntax is equivalent to:: return func func = TYPE_PARAMS_OF_func() -Here ``def'`` indicates an :ref:`annotation scope `. (Two other -liberties are taken in the translation: the ``__type_params__`` -attribute of generic functions is not writable from Python code; -and the syntax does not go through attribute access on the :mod:`typing` -module, but creates an instance of :data:`typing.TypeVar` directly.) +Here ``def'`` indicates an :ref:`annotation scope `. (One +other liberty is taken in the translation: the syntax does not go through +attribute access on the :mod:`typing` module, but creates an instance of +:data:`typing.TypeVar` directly.) -The annotations of generic functions are defined within the annotation scope +The annotations of generic functions are evaluated within the annotation scope used for declaring the type parameters, but the function's defaults and decorators are not. @@ -1770,8 +1783,8 @@ Here again ``def'`` indicates an :ref:`annotation scope `. Generic classes implicitly inherit from :data:`typing.Generic`. The base classes and keyword arguments of generic classes are -executed within the type scope for the type parameters, -and decorators are executed outside that scope. This is illustrated +evaluated within the type scope for the type parameters, +and decorators are evaluated outside that scope. This is illustrated by this example:: @decorator From 12c1249b087f9d8f819b191b8cb10fa942d2d5e7 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:11:25 -0700 Subject: [PATCH 08/56] minor --- Doc/reference/executionmodel.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index a37fd8281e2c30..81751da2b29729 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -197,7 +197,7 @@ annotation scopes in Python 3.13 when :pep:`649` is implemented. Annotation scopes are used in the following contexts: -* Type parameter lists for generic type aliases +* Type parameter lists for generic type aliases. * Type parameter lists for generic functions. The function's annotations are executed within the type scope, but its defaults and decorators are not. * Type parameter lists for generic classes. The class's base classes and @@ -212,7 +212,7 @@ Annotation scopes differ from function scopes in the following ways: class scope can be accessed from within the annotation scope, as if the code in the annotation scope was executing directly within the class scope. (By contrast, regular functions defined within classes cannot access names defined in the class scope.) -* Expressions in annotation scopes cannot contain :keyword:`yield`, ``yield from ``, +* Expressions in annotation scopes cannot contain :keyword:`yield`, ``yield from``, :keyword:`await`, or :token:`:= <~python-grammar:expression>` expressions. (These expressions are allowed in other scopes contained within the type scope.) * Names defined in annotation scopes cannot be rebound with :keyword:`nonlocal` From 94a4da2a15c311aee9b2a179959b6534eab60036 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:12:56 -0700 Subject: [PATCH 09/56] Remove broken link --- Doc/reference/executionmodel.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 81751da2b29729..376918403f7939 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -219,7 +219,7 @@ Annotation scopes differ from function scopes in the following ways: statements in inner scopes. This includes only type parameters, as no other syntactic elements that can appear within type scopes can introduce new names. * While annotation scopes internally have a name, that name is not reflected in the - :attr:`__qualname__` of objects defined within the scope. Instead, the :attr:`!__qualname__` + ``__qualname__`` of objects defined within the scope. Instead, the ``__qualname__`` of such objects is as if the object was defined in the enclosing scope. .. versionadded:: 3.12 From 815a0219376092c76b0c8ef08e462b3c159b0f98 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:14:31 -0700 Subject: [PATCH 10/56] update dis.dis docs --- Doc/library/dis.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 2a1f1704827b2e..6174f55aaddba6 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -189,8 +189,8 @@ operation is being performed, so the intermediate analysis object isn't useful: all methods (including class and static methods). For a code object or sequence of raw bytecode, it prints one line per bytecode instruction. It also recursively disassembles nested code objects (the code of - comprehensions, generator expressions and nested functions, and the code - used for building nested classes). + generator expressions and nested functions, and the code + used for building nested classes and :ref:`annotation scopes `). Strings are first compiled to code objects with the :func:`compile` built-in function before being disassembled. If no object is provided, this function disassembles the last traceback. From 17ac78543eb175a82c18bbe79289f31c1cd17bf0 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:16:32 -0700 Subject: [PATCH 11/56] TypeAlias is deprecated --- Doc/library/typing.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index c5cba8fe2935c3..2ae12be5a1d17f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -754,6 +754,9 @@ These can be used as types in annotations and do not support ``[]``. .. versionadded:: 3.10 + .. deprecated:: 3.12 + :data:`TypeAlias` is deprecated in favor of the :keyword:`type` statement. + Special forms """"""""""""" @@ -2996,3 +2999,5 @@ convenience. This is subject to change, and not all deprecations are listed. | ``typing.Hashable`` and | 3.12 | Undecided | :gh:`94309` | | ``typing.Sized`` | | | | +----------------------------------+---------------+-------------------+----------------+ +| ``typing.TypeAlias`` | 3.12 | Undecided | :pep`695` | ++----------------------------------+---------------+-------------------+----------------+ From 1311399d12e9d2693511ace193f90432674f70b5 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:22:41 -0700 Subject: [PATCH 12/56] Type aliases should now use the type statement --- Doc/library/typing.rst | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 2ae12be5a1d17f..d8eeb11d45bc02 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -111,10 +111,10 @@ annotations. These include: Type aliases ============ -A type alias is defined by assigning the type to the alias. In this example, +A type alias is defined using the :keyword:`type` statement. In this example, ``Vector`` and ``list[float]`` will be treated as interchangeable synonyms:: - Vector = list[float] + type Vector = list[float] def scale(scalar: float, vector: Vector) -> Vector: return [scalar * num for num in vector] @@ -126,9 +126,9 @@ Type aliases are useful for simplifying complex type signatures. For example:: from collections.abc import Sequence - ConnectionOptions = dict[str, str] - Address = tuple[str, int] - Server = tuple[Address, ConnectionOptions] + type ConnectionOptions = dict[str, str] + type Address = tuple[str, int] + type Server = tuple[Address, ConnectionOptions] def broadcast_message(message: str, servers: Sequence[Server]) -> None: ... @@ -143,6 +143,17 @@ Type aliases are useful for simplifying complex type signatures. For example:: Note that ``None`` as a type hint is a special case and is replaced by ``type(None)``. +The :keyword:`type` statement is new in Python 3.12. For backwards +compatibility, type aliases can also created through simple assignment:: + + Vector = list[float] + +Or marked with :data:`TypeAlias`:: + + from typing import TypeAlias + + Vector: TypeAlias = list[float] + .. _distinct: NewType From 123752ca0d4093fc949cd61334c50a4d28f036af Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 18 May 2023 22:32:10 -0700 Subject: [PATCH 13/56] syntax --- Doc/library/typing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d8eeb11d45bc02..a3fc7be8410468 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -3010,5 +3010,5 @@ convenience. This is subject to change, and not all deprecations are listed. | ``typing.Hashable`` and | 3.12 | Undecided | :gh:`94309` | | ``typing.Sized`` | | | | +----------------------------------+---------------+-------------------+----------------+ -| ``typing.TypeAlias`` | 3.12 | Undecided | :pep`695` | +| ``typing.TypeAlias`` | 3.12 | Undecided | :pep:`695` | +----------------------------------+---------------+-------------------+----------------+ From e955a459e6ca5c674e456e08491786465eb92dd3 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 19 May 2023 19:35:14 -0700 Subject: [PATCH 14/56] Expand typing.rst --- Doc/library/typing.rst | 495 ++++++++++++++++++++++++++++------------- 1 file changed, 335 insertions(+), 160 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index a3fc7be8410468..e797e7164a951f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -111,7 +111,8 @@ annotations. These include: Type aliases ============ -A type alias is defined using the :keyword:`type` statement. In this example, +A type alias is defined using the :keyword:`type` statement, which creates +an instance of :class:`TypeAliasType`. In this example, ``Vector`` and ``list[float]`` will be treated as interchangeable synonyms:: type Vector = list[float] @@ -144,7 +145,7 @@ Note that ``None`` as a type hint is a special case and is replaced by ``type(None)``. The :keyword:`type` statement is new in Python 3.12. For backwards -compatibility, type aliases can also created through simple assignment:: +compatibility, type aliases can also be created through simple assignment:: Vector = list[float] @@ -295,10 +296,14 @@ subscription to denote expected types for container elements. def notify_by_email(employees: Sequence[Employee], overrides: Mapping[str, str]) -> None: ... -Generics can be parameterized by using a factory available in typing -called :class:`TypeVar`. +Generics can be parameterized by using :ref:`type parameter syntax `:: -:: + from collections.abc import Sequence + + def first[T](l: Sequence[T]) -> T: # Generic function + return l[0] + +Or by using the :class:`TypeVar` factory directly:: from collections.abc import Sequence from typing import TypeVar @@ -308,6 +313,9 @@ called :class:`TypeVar`. def first(l: Sequence[T]) -> T: # Generic function return l[0] +.. versionchanged:: 3.12 + Syntactic support for generics is new in version 3.12. + .. _user-defined-generics: User-defined generic types @@ -317,12 +325,9 @@ A user-defined class can be defined as a generic class. :: - from typing import TypeVar, Generic from logging import Logger - T = TypeVar('T') - - class LoggedVar(Generic[T]): + class LoggedVar[T]: def __init__(self, value: T, name: str, logger: Logger) -> None: self.name = name self.logger = logger @@ -339,10 +344,21 @@ A user-defined class can be defined as a generic class. def log(self, message: str) -> None: self.logger.info('%s: %s', self.name, message) -``Generic[T]`` as a base class defines that the class ``LoggedVar`` takes a +This syntax indicates that the class ``LoggedVar`` takes a single type parameter ``T`` . This also makes ``T`` valid as a type within the class body. +Generic classes implicitly inherit from :class:`Generic`. For compatibility +with Python 3.11 and lower, it is also possible to inherit explicitly from +:class:`Generic`:: + + from typing import TypeVar, Generic + + T = TypeVar('T') + + class LoggedVar(Generic[T]): + ... + The :class:`Generic` base class defines :meth:`~object.__class_getitem__` so that ``LoggedVar[T]`` is valid as a type:: @@ -361,7 +377,10 @@ A generic type can have any number of type variables. All varieties of B = TypeVar('B', bound=Sequence[bytes], covariant=True) S = TypeVar('S', int, str) - class WeirdTrio(Generic[T, B, S]): + class OldWeirdTrio(Generic[T, B, S]): + ... + + class NewWeirdTrio[T, B: Sequence[bytes], S: (int, str)]: ... Each type variable argument to :class:`Generic` must be distinct. @@ -370,29 +389,26 @@ This is thus invalid:: from typing import TypeVar, Generic ... + class Pair[T, T]: # SyntaxError + ... + T = TypeVar('T') class Pair(Generic[T, T]): # INVALID ... -You can use multiple inheritance with :class:`Generic`:: +Generic classes can also inherit from other classes:: from collections.abc import Sized - from typing import TypeVar, Generic - T = TypeVar('T') - - class LinkedList(Sized, Generic[T]): + class LinkedList[T](Sized): ... When inheriting from generic classes, some type variables could be fixed:: from collections.abc import Mapping - from typing import TypeVar - T = TypeVar('T') - - class MyDict(Mapping[str, T]): + class MyDict[T](Mapping[str, T]): ... In this case ``MyDict`` has a single parameter, ``T``. @@ -408,46 +424,62 @@ not generic but implicitly inherits from ``Iterable[Any]``:: User defined generic type aliases are also supported. Examples:: from collections.abc import Iterable - from typing import TypeVar - S = TypeVar('S') - Response = Iterable[S] | int + + type Response[S] = Iterable[S] | int # Return type here is same as Iterable[str] | int def response(query: str) -> Response[str]: ... - T = TypeVar('T', int, float, complex) - Vec = Iterable[tuple[T, T]] + type Vec[T] = Iterable[tuple[T, T]] - def inproduct(v: Vec[T]) -> T: # Same as Iterable[tuple[T, T]] + def inproduct[T: (int, float, complex)](v: Vec[T]) -> T: # Same as Iterable[tuple[T, T]] return sum(x*y for x, y in v) +For backward compatibility, generic type aliases can also be created +through a simple assignment:: + + from collections.abc import Iterable + from typing import TypeVar + + S = TypeVar("S") + Response = Iterable[S] | int + .. versionchanged:: 3.7 :class:`Generic` no longer has a custom metaclass. +.. versionchanged:: 3.12 + Syntactic support for generics and type aliases is new in version 3.12. + Previously, generic classes had to inherit from :class:`Generic` + or contain a type variable in one of their bases. + User-defined generics for parameter expressions are also supported via parameter -specification variables in the form ``Generic[P]``. The behavior is consistent +specification variables in the form ``[**P]``. The behavior is consistent with type variables' described above as parameter specification variables are treated by the typing module as a specialized type variable. The one exception to this is that a list of types can be used to substitute a :class:`ParamSpec`:: - >>> from typing import Generic, ParamSpec, TypeVar - - >>> T = TypeVar('T') - >>> P = ParamSpec('P') - - >>> class Z(Generic[T, P]): ... + >>> class Z[T, **P]: ... # P is a ParamSpec ... >>> Z[int, [dict, float]] __main__.Z[int, [dict, float]] +Classes generic over a :class:`ParamSpec` can also be created using explicit +inheritance from :class:`Generic`. In this case, ``**`` is not used:: + + from typing import ParamSpec, Generic + + P = ParamSpec('P') + + class Z(Generic[P]): + ... Furthermore, a generic with only one parameter specification variable will accept parameter lists in the forms ``X[[Type1, Type2, ...]]`` and also ``X[Type1, Type2, ...]`` for aesthetic reasons. Internally, the latter is converted to the former, so the following are equivalent:: - >>> class X(Generic[P]): ... + >>> class X[**P]: ... ... >>> X[int, str] __main__.X[[int, str]] @@ -629,6 +661,21 @@ These can be used as types in annotations and do not support ``[]``. avoiding type checker errors with classes that can duck type anywhere or are highly dynamic. +.. data:: AnyStr + + ``AnyStr`` is a :ref:`constrained type variable ` defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. + + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + .. data:: LiteralString Special type that includes only literal strings. A string @@ -766,7 +813,11 @@ These can be used as types in annotations and do not support ``[]``. .. versionadded:: 3.10 .. deprecated:: 3.12 - :data:`TypeAlias` is deprecated in favor of the :keyword:`type` statement. + :data:`TypeAlias` is deprecated in favor of the :keyword:`type` statement, + which creates instances of :class:`TypeAliasType`. + Note that while :data:`TypeAlias` and :class:`TypeAliasType` serve + similar purposes and have similar names, they are distinct and the + latter is not the type of the former. Special forms """"""""""""" @@ -942,7 +993,7 @@ These can be used as types in annotations using ``[]``, each having a unique syn * :class:`ParamSpec` and :class:`Callable`. -.. class:: Type(Generic[CT_co]) +.. class:: Type[CT_co] A variable annotated with ``C`` may accept a value of type ``C``. In contrast, a variable annotated with ``Type[C]`` may accept values that are @@ -1223,36 +1274,93 @@ These can be used as types in annotations using ``[]``, each having a unique syn .. versionadded:: 3.10 +.. data:: Unpack + + A typing operator that conceptually marks an object as having been + unpacked. For example, using the unpack operator ``*`` on a + :class:`type variable tuple ` is equivalent to using ``Unpack`` + to mark the type variable tuple as having been unpacked:: + + Ts = TypeVarTuple('Ts') + tup: tuple[*Ts] + # Effectively does: + tup: tuple[Unpack[Ts]] + + In fact, ``Unpack`` can be used interchangeably with ``*`` in the context + of :class:`typing.TypeVarTuple ` and + :class:`builtins.tuple ` types. You might see ``Unpack`` being used + explicitly in older versions of Python, where ``*`` couldn't be used in + certain places:: + + # In older versions of Python, TypeVarTuple and Unpack + # are located in the `typing_extensions` backports package. + from typing_extensions import TypeVarTuple, Unpack + + Ts = TypeVarTuple('Ts') + tup: tuple[*Ts] # Syntax error on Python <= 3.10! + tup: tuple[Unpack[Ts]] # Semantically equivalent, and backwards-compatible + + ``Unpack`` can also be used along with :class:`typing.TypedDict` for typing + ``**kwargs`` in a function signature:: + + from typing import TypedDict, Unpack + + class Movie(TypedDict): + name: str + year: int + + # This function expects two keyword arguments - `name` of type `str` + # and `year` of type `int`. + def foo(**kwargs: Unpack[Movie]): ... + + See :pep:`692` for more details on using ``Unpack`` for ``**kwargs`` typing. -Building generic types -"""""""""""""""""""""" + .. versionadded:: 3.11 + +Building generic types and type aliases +""""""""""""""""""""""""""""""""""""""" These are not used in annotations. They are building blocks for creating generic types. +They can be created through special syntax (:ref:`type parameter lists ` +and the :keyword:`type` statement). For compatibility with Python 3.11 and earlier, they +can also be created without the dedicated syntax, as documented below. .. class:: Generic Abstract base class for generic types. - A generic type is typically declared by inheriting from an - instantiation of this class with one or more type variables. - For example, a generic mapping type might be defined as:: + A generic type is typically declared by adding a list of type parameters + after the class name:: - class Mapping(Generic[KT, VT]): + class Mapping[KT, VT]: def __getitem__(self, key: KT) -> VT: ... # Etc. - This class can then be used as follows:: + Such a class implicitly inherits from ``Generic``. + The runtime semantics of this syntax are discussed in the + :ref:`Language Reference `. - X = TypeVar('X') - Y = TypeVar('Y') + This class can then be used as follows:: - def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: + def lookup_name[X, Y](mapping: Mapping[X, Y], key: X, default: Y) -> Y: try: return mapping[key] except KeyError: return default + Generic classes can also be declared by explicitly inheriting from + ``Generic``. In this case, the type parameters must be created + separately:: + + KT = TypeVar('KT') + VT = TypeVar('VT') + + class Mapping(Generic[KT, VT]): + def __getitem__(self, key: KT) -> VT: + ... + # Etc. + .. class:: TypeVar Type variable. @@ -1263,23 +1371,41 @@ These are not used in annotations. They are building blocks for creating generic S = TypeVar('S', bound=str) # Can be any subtype of str A = TypeVar('A', str, bytes) # Must be exactly str or bytes + The syntax for :ref:`generic functions `, + :ref:`generic classes `, and + :ref:`generic type aliases ` can be used to + create type variables:: + + class Sequence[T]: # T is a TypeVar + ... + + This syntax can also be used to create bound and constrained type + variables:: + + class StrSequence[S: str]: # S is a TypeVar bound to str + ... + + + class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes + ... + Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See :class:`Generic` for more information on generic types. Generic functions work as follows:: - def repeat(x: T, n: int) -> Sequence[T]: + def repeat[T](x: T, n: int) -> Sequence[T]: """Return a list containing n references to x.""" return [x]*n - def print_capitalized(x: S) -> S: + def print_capitalized[S: str](x: S) -> S: """Print x capitalized, and return x.""" print(x.capitalize()) return x - def concatenate(x: A, y: A) -> A: + def concatenate[A: (str, bytes)](x: A, y: A) -> A: """Add two strings or bytes objects together.""" return x + y @@ -1307,6 +1433,10 @@ These are not used in annotations. They are building blocks for creating generic U = TypeVar('U', bound=str|bytes) # Can be any subtype of the union str|bytes V = TypeVar('V', bound=SupportsAbs) # Can be anything with an __abs__ method + # Can be anything with an __abs__ method + def print_abs[V: SupportsAbs](arg: V) -> None: + print("Absolute value:", abs(arg)) + .. _typing-constrained-typevar: Using a *constrained* type variable, however, means that the ``TypeVar`` @@ -1327,22 +1457,34 @@ These are not used in annotations. They are building blocks for creating generic ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more details. By default, type variables are invariant. + .. versionchanged:: 3.12 + + Type variables can now be declared using the + :ref:`type parameter ` syntax introduced by :pep:`695`. + .. class:: TypeVarTuple Type variable tuple. A specialized form of :class:`type variable ` that enables *variadic* generics. - A normal type variable enables parameterization with a single type. A type - variable tuple, in contrast, allows parameterization with an - *arbitrary* number of types by acting like an *arbitrary* number of type - variables wrapped in a tuple. For example:: + Type variable tuples can be declared in :ref:`type parameter lists ` + using a single asterisk (``*``) before the name:: + + def move_first_element_to_last[T, *Ts](tup: tuple[T, *Ts]) -> tuple[*Ts, T]: + return (*tup[1:], tup[0]) - T = TypeVar('T') - Ts = TypeVarTuple('Ts') + Or by explicitly invoking the ``TypeVarTuple`` constructor:: + T = TypeVar("T") + Ts = TypeVarTuple("Ts") def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]: return (*tup[1:], tup[0]) + A normal type variable enables parameterization with a single type. A type + variable tuple, in contrast, allows parameterization with an + *arbitrary* number of types by acting like an *arbitrary* number of type + variables wrapped in a tuple. For example:: + # T is bound to int, Ts is bound to () # Return value is (1,), which has type tuple[int] move_first_element_to_last(tup=(1,)) @@ -1378,8 +1520,7 @@ These are not used in annotations. They are building blocks for creating generic Type variable tuples can be used in the same contexts as normal type variables. For example, in class definitions, arguments, and return types:: - Shape = TypeVarTuple('Shape') - class Array(Generic[*Shape]): + class Array[*Shape]: def __getitem__(self, key: tuple[*Shape]) -> float: ... def __abs__(self) -> "Array[*Shape]": ... def get_shape(self) -> tuple[*Shape]: ... @@ -1388,10 +1529,10 @@ These are not used in annotations. They are building blocks for creating generic DType = TypeVar('DType') - class Array(Generic[DType, *Shape]): # This is fine + class Array[DType, *Shape]: # This is fine pass - class Array2(Generic[*Shape, DType]): # This would also be fine + class Array2[*Shape, DType]: # This would also be fine pass float_array_1d: Array[float, Height] = Array() # Totally fine @@ -1401,13 +1542,13 @@ These are not used in annotations. They are building blocks for creating generic list of type arguments or type parameters:: x: tuple[*Ts, *Ts] # Not valid - class Array(Generic[*Shape, *Shape]): # Not valid + class Array[*Shape, *Shape]: # Not valid pass Finally, an unpacked type variable tuple can be used as the type annotation of ``*args``:: - def call_soon( + def call_soon[*Ts]( callback: Callable[[*Ts], None], *args: *Ts ) -> None: @@ -1425,55 +1566,23 @@ These are not used in annotations. They are building blocks for creating generic .. versionadded:: 3.11 -.. data:: Unpack - - A typing operator that conceptually marks an object as having been - unpacked. For example, using the unpack operator ``*`` on a - :class:`type variable tuple ` is equivalent to using ``Unpack`` - to mark the type variable tuple as having been unpacked:: - - Ts = TypeVarTuple('Ts') - tup: tuple[*Ts] - # Effectively does: - tup: tuple[Unpack[Ts]] - - In fact, ``Unpack`` can be used interchangeably with ``*`` in the context - of :class:`typing.TypeVarTuple ` and - :class:`builtins.tuple ` types. You might see ``Unpack`` being used - explicitly in older versions of Python, where ``*`` couldn't be used in - certain places:: - - # In older versions of Python, TypeVarTuple and Unpack - # are located in the `typing_extensions` backports package. - from typing_extensions import TypeVarTuple, Unpack - - Ts = TypeVarTuple('Ts') - tup: tuple[*Ts] # Syntax error on Python <= 3.10! - tup: tuple[Unpack[Ts]] # Semantically equivalent, and backwards-compatible - - ``Unpack`` can also be used along with :class:`typing.TypedDict` for typing - ``**kwargs`` in a function signature:: - - from typing import TypedDict, Unpack - - class Movie(TypedDict): - name: str - year: int - - # This function expects two keyword arguments - `name` of type `str` - # and `year` of type `int`. - def foo(**kwargs: Unpack[Movie]): ... - - See :pep:`692` for more details on using ``Unpack`` for ``**kwargs`` typing. + .. versionchanged:: 3.12 - .. versionadded:: 3.11 + Type variable tuples can now be declared using the + :ref:`type parameter ` syntax introduced by :pep:`695`. .. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False) Parameter specification variable. A specialized version of :class:`type variables `. - Usage:: + In :ref:`type parameter lists `, parameter specifications + can be declared with two asterisks (``**``):: + + type IntFunc[**P] = Callable[P, int] + + For compatibility with Python 3.11 and earlier, ``ParamSpec`` objects + can also be created as follows:: P = ParamSpec('P') @@ -1490,13 +1599,9 @@ These are not used in annotations. They are building blocks for creating generic new callable returned by it have inter-dependent type parameters:: from collections.abc import Callable - from typing import TypeVar, ParamSpec import logging - T = TypeVar('T') - P = ParamSpec('P') - - def add_logging(f: Callable[P, T]) -> Callable[P, T]: + def add_logging[T, **P](f: Callable[P, T]) -> Callable[P, T]: '''A type-safe decorator to add logging to a function.''' def inner(*args: P.args, **kwargs: P.kwargs) -> T: logging.info(f'{f.__name__} was called') @@ -1539,6 +1644,11 @@ These are not used in annotations. They are building blocks for creating generic .. versionadded:: 3.10 + .. versionchanged:: 3.12 + + Parameter specifications can now be declared using the + :ref:`type parameter ` syntax introduced by :pep:`695`. + .. note:: Only parameter specification variables defined in global scope can be pickled. @@ -1566,20 +1676,71 @@ These are not used in annotations. They are building blocks for creating generic .. versionadded:: 3.10 -.. data:: AnyStr +.. class:: TypeAliasType(name, value, *, type_params=()) - ``AnyStr`` is a :ref:`constrained type variable ` defined as - ``AnyStr = TypeVar('AnyStr', str, bytes)``. + The type of type aliases created through the :keyword:`type` statement. - It is meant to be used for functions that may accept any kind of string - without allowing different kinds of strings to mix. For example:: + Example:: - def concat(a: AnyStr, b: AnyStr) -> AnyStr: - return a + b + >>> type Alias = int + >>> type(Alias) + - concat(u"foo", u"bar") # Ok, output has type 'unicode' - concat(b"foo", b"bar") # Ok, output has type 'bytes' - concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + .. versionadded:: 3.12 + + .. attribute:: __name__ + + The name of the type alias. + + Example:: + + >>> type Alias = int + >>> Alias.__name__ + 'Alias' + + .. attribute:: __module__ + + The module in which the type alias was defined. + + Example:: + + >>> type Alias = int + >>> Alias.__module__ + '__main__' + + .. attribute:: __type_params__ + + The type parameters of the type alias, or an empty tuple if the alias is + not generic. + + Example:: + + >>> type ListOrSet[T] = list[T] | set[T] + >>> Alias.__type_params__ + (T,) + >>> type NotGeneric = int + >>> NotGeneric.__type_params__ + () + + .. attribute:: __value__ + + The type alias's value. This is lazily evaluated, so names used in the + definition of the alias are not resolved until the ``__value__`` attribute + is accessed. + + Example:: + + >>> type Mutually = Recursive + >>> type Recursive = Mutually + >>> Mutually.__value__ + Recursive + >>> Recursive.__value__ + Mutually + +Other special directives +"""""""""""""""""""""""" + +These are not used in annotations. They are building blocks for declaring types. .. class:: Protocol(Generic) @@ -1608,6 +1769,15 @@ These are not used in annotations. They are building blocks for creating generic Protocol classes can be generic, for example:: + class GenProto[T](Protocol): + def meth(self) -> T: + ... + + In code that needs to be compatible with Python 3.11 or older, generic + Protocols can be written as follows:: + + T = TypeVar("T") + class GenProto(Protocol[T]): def meth(self) -> T: ... @@ -1674,12 +1844,6 @@ These are not used in annotations. They are building blocks for creating generic protocol. See :ref:`"What's new in Python 3.12" ` for more details. - -Other special directives -"""""""""""""""""""""""" - -These are not used in annotations. They are building blocks for declaring types. - .. class:: NamedTuple Typed version of :func:`collections.namedtuple`. @@ -1723,12 +1887,18 @@ These are not used in annotations. They are building blocks for declaring types. ``NamedTuple`` subclasses can be generic:: - class Group(NamedTuple, Generic[T]): + class Group[T](NamedTuple): key: T group: list[T] Backward-compatible usage:: + # For creating a generic NamedTuple on Python 3.11 or lower + class Group(NamedTuple, Generic[T]): + key: T + group: list[T] + + # For creating a NamedTuple on Python 3.5 or lower Employee = NamedTuple('Employee', [('name', str), ('id', int)]) .. versionchanged:: 3.6 @@ -1891,6 +2061,13 @@ These are not used in annotations. They are building blocks for declaring types. A ``TypedDict`` can be generic:: + class Group[T](TypedDict): + key: T + group: list[T] + + To create a generic ``TypedDict`` that is compatible with Python 3.11 + or lower, use :class:`Generic` explicitly:: + class Group(TypedDict, Generic[T]): key: T group: list[T] @@ -1965,7 +2142,7 @@ Generic concrete collections Corresponding to built-in types """"""""""""""""""""""""""""""" -.. class:: Dict(dict, MutableMapping[KT, VT]) +.. class:: Dict[KT, VT](dict, MutableMapping[KT, VT]) A generic version of :class:`dict`. Useful for annotating return types. To annotate arguments it is preferred @@ -1980,7 +2157,7 @@ Corresponding to built-in types :class:`builtins.dict ` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: List(list, MutableSequence[T]) +.. class:: List[T](list, MutableSequence[T]) Generic version of :class:`list`. Useful for annotating return types. To annotate arguments it is preferred @@ -1989,19 +2166,17 @@ Corresponding to built-in types This type may be used as follows:: - T = TypeVar('T', int, float) - - def vec2(x: T, y: T) -> List[T]: + def vec2[T: (int, float)](x: T, y: T) -> List[T]: return [x, y] - def keep_positives(vector: Sequence[T]) -> List[T]: + def keep_positives[T: (int, float)](vector: Sequence[T]) -> List[T]: return [item for item in vector if item > 0] .. deprecated:: 3.9 :class:`builtins.list ` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Set(set, MutableSet[T]) +.. class:: Set[T](set, MutableSet[T]) A generic version of :class:`builtins.set `. Useful for annotating return types. To annotate arguments it is preferred @@ -2011,7 +2186,7 @@ Corresponding to built-in types :class:`builtins.set ` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) +.. class:: FrozenSet[T_co](frozenset, AbstractSet[T_co]) A generic version of :class:`builtins.frozenset `. @@ -2025,7 +2200,7 @@ Corresponding to built-in types Corresponding to types in :mod:`collections` """""""""""""""""""""""""""""""""""""""""""" -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) +.. class:: DefaultDict[KT, VT](collections.defaultdict, MutableMapping[KT, VT]) A generic version of :class:`collections.defaultdict`. @@ -2035,7 +2210,7 @@ Corresponding to types in :mod:`collections` :class:`collections.defaultdict` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) +.. class:: OrderedDict[KT, VT](collections.OrderedDict, MutableMapping[KT, VT]) A generic version of :class:`collections.OrderedDict`. @@ -2045,7 +2220,7 @@ Corresponding to types in :mod:`collections` :class:`collections.OrderedDict` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) +.. class:: ChainMap[KT, VT](collections.ChainMap, MutableMapping[KT, VT]) A generic version of :class:`collections.ChainMap`. @@ -2056,7 +2231,7 @@ Corresponding to types in :mod:`collections` :class:`collections.ChainMap` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Counter(collections.Counter, Dict[T, int]) +.. class:: Counter[T](collections.Counter, Dict[T, int]) A generic version of :class:`collections.Counter`. @@ -2067,7 +2242,7 @@ Corresponding to types in :mod:`collections` :class:`collections.Counter` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Deque(deque, MutableSequence[T]) +.. class:: Deque[T](deque, MutableSequence[T]) A generic version of :class:`collections.deque`. @@ -2138,7 +2313,7 @@ Abstract Base Classes Corresponding to collections in :mod:`collections.abc` """""""""""""""""""""""""""""""""""""""""""""""""""""" -.. class:: AbstractSet(Collection[T_co]) +.. class:: AbstractSet[T_co](Collection[T_co]) A generic version of :class:`collections.abc.Set`. @@ -2154,7 +2329,7 @@ Corresponding to collections in :mod:`collections.abc` .. deprecated-removed:: 3.9 3.14 Prefer :class:`collections.abc.Buffer`, or a union like ``bytes | bytearray | memoryview``. -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) +.. class:: Collection[T_co](Sized, Iterable[T_co], Container[T_co]) A generic version of :class:`collections.abc.Collection` @@ -2164,7 +2339,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Collection` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Container(Generic[T_co]) +.. class:: Container[T_co] A generic version of :class:`collections.abc.Container`. @@ -2172,7 +2347,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Container` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ItemsView(MappingView, AbstractSet[tuple[KT_co, VT_co]]) +.. class:: ItemsView[KT_co, VT_co](MappingView, AbstractSet[tuple[KT_co, VT_co]]) A generic version of :class:`collections.abc.ItemsView`. @@ -2180,7 +2355,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.ItemsView` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: KeysView(MappingView, AbstractSet[KT_co]) +.. class:: KeysView[KT_co](MappingView, AbstractSet[KT_co]) A generic version of :class:`collections.abc.KeysView`. @@ -2188,7 +2363,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.KeysView` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Mapping(Collection[KT], Generic[KT, VT_co]) +.. class:: Mapping[KT, VT_co](Collection[KT]) A generic version of :class:`collections.abc.Mapping`. This type can be used as follows:: @@ -2208,7 +2383,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.MappingView` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: MutableMapping(Mapping[KT, VT]) +.. class:: MutableMapping[KT, VT](Mapping[KT, VT]) A generic version of :class:`collections.abc.MutableMapping`. @@ -2217,7 +2392,7 @@ Corresponding to collections in :mod:`collections.abc` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: MutableSequence(Sequence[T]) +.. class:: MutableSequence[T](Sequence[T]) A generic version of :class:`collections.abc.MutableSequence`. @@ -2226,7 +2401,7 @@ Corresponding to collections in :mod:`collections.abc` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: MutableSet(AbstractSet[T]) +.. class:: MutableSet[T](AbstractSet[T]) A generic version of :class:`collections.abc.MutableSet`. @@ -2234,7 +2409,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.MutableSet` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Sequence(Reversible[T_co], Collection[T_co]) +.. class:: Sequence[T_co](Reversible[T_co], Collection[T_co]) A generic version of :class:`collections.abc.Sequence`. @@ -2242,7 +2417,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Sequence` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ValuesView(MappingView, Collection[_VT_co]) +.. class:: ValuesView[VT_co](MappingView, Collection[VT_co]) A generic version of :class:`collections.abc.ValuesView`. @@ -2253,7 +2428,7 @@ Corresponding to collections in :mod:`collections.abc` Corresponding to other types in :mod:`collections.abc` """""""""""""""""""""""""""""""""""""""""""""""""""""" -.. class:: Iterable(Generic[T_co]) +.. class:: Iterable[T_co] A generic version of :class:`collections.abc.Iterable`. @@ -2261,7 +2436,7 @@ Corresponding to other types in :mod:`collections.abc` :class:`collections.abc.Iterable` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Iterator(Iterable[T_co]) +.. class:: Iterator[T_co](Iterable[T_co]) A generic version of :class:`collections.abc.Iterator`. @@ -2269,7 +2444,7 @@ Corresponding to other types in :mod:`collections.abc` :class:`collections.abc.Iterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) +.. class:: Generator[YieldType, SendType, ReturnType](Iterator[YieldType]) A generator can be annotated by the generic type ``Generator[YieldType, SendType, ReturnType]``. For example:: @@ -2311,7 +2486,7 @@ Corresponding to other types in :mod:`collections.abc` .. deprecated:: 3.12 Use :class:`collections.abc.Hashable` directly instead. -.. class:: Reversible(Iterable[T_co]) +.. class:: Reversible[T_co](Iterable[T_co]) A generic version of :class:`collections.abc.Reversible`. @@ -2329,7 +2504,7 @@ Corresponding to other types in :mod:`collections.abc` Asynchronous programming """""""""""""""""""""""" -.. class:: Coroutine(Awaitable[V_co], Generic[T_co, T_contra, V_co]) +.. class:: Coroutine[YieldType, SendType, ReturnType](Awaitable[ReturnType]) A generic version of :class:`collections.abc.Coroutine`. The variance and order of type variables @@ -2347,7 +2522,7 @@ Asynchronous programming :class:`collections.abc.Coroutine` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) +.. class:: AsyncGenerator[YieldType, SendType](AsyncIterator[YieldType]) An async generator can be annotated by the generic type ``AsyncGenerator[YieldType, SendType]``. For example:: @@ -2385,7 +2560,7 @@ Asynchronous programming now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncIterable(Generic[T_co]) +.. class:: AsyncIterable[T_co] A generic version of :class:`collections.abc.AsyncIterable`. @@ -2395,7 +2570,7 @@ Asynchronous programming :class:`collections.abc.AsyncIterable` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncIterator(AsyncIterable[T_co]) +.. class:: AsyncIterator[T_co](AsyncIterable[T_co]) A generic version of :class:`collections.abc.AsyncIterator`. @@ -2405,7 +2580,7 @@ Asynchronous programming :class:`collections.abc.AsyncIterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Awaitable(Generic[T_co]) +.. class:: Awaitable[T_co] A generic version of :class:`collections.abc.Awaitable`. @@ -2419,7 +2594,7 @@ Asynchronous programming Context manager types """"""""""""""""""""" -.. class:: ContextManager(Generic[T_co]) +.. class:: ContextManager[T_co] A generic version of :class:`contextlib.AbstractContextManager`. @@ -2431,7 +2606,7 @@ Context manager types now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncContextManager(Generic[T_co]) +.. class:: AsyncContextManager[T_co] A generic version of :class:`contextlib.AbstractAsyncContextManager`. From d7f2b55dd8cc2bfe2246a1c4af186eb4023805ac Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 19 May 2023 21:05:07 -0700 Subject: [PATCH 15/56] Finish the type statement --- Doc/reference/simple_stmts.rst | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 228ea795548903..902d969ce4e272 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -1024,9 +1024,28 @@ The :keyword:`!type` statement .. productionlist:: python-grammar type_stmt: 'type' `identifier` [`type_params`] "=" `expression` -The :keyword:`!type` statement introduces a type alias. +The :keyword:`!type` statement introduces a type alias, which is an instance +of :class:`typing.TypeAliasType`. -TODO(jelle): Link to typing.TypeAliasType. Link to section on type params. Link to section on lazy eval of RHS. +For example, the following statement creates a type alias:: + + type Point = tuple[float, float] + +This code is roughly equivalent to:: + + def' VALUE_OF_Point(): + return tuple[float, float] + Point = typing.TypeAliasType("Point", VALUE_OF_Point()) + +``def'`` indicates an :ref:`annotation scope `, which behaves +mostly like a function, but with several small differences. The value of the +type alias is evaluated in the annotation scope. It is not evaluated when the +type alias is created, but only when the value is accessed through the type alias's +``__value__`` attribute. This allows the type alias to refer to names that are +not yet defined. + +Type aliases may be made generic by adding a :ref:`type parameter list ` +after the name. See :ref:`generic-type-aliases` for more. :keyword:`!type` is a :ref:`soft keyword `. From 53c49165a9e88acb3a8bf788b0a540146381cabc Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 19 May 2023 21:05:15 -0700 Subject: [PATCH 16/56] What's New --- Doc/whatsnew/3.12.rst | 64 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index aeb534afb0ceb6..a1307bf0d04a04 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -70,6 +70,8 @@ New typing features: * :pep:`688`: Making the buffer protocol accessible in Python +* :ref:`whatsnew312-pep695` + * :ref:`whatsnew312-pep692` * :pep:`698`: Override Decorator for Static Typing @@ -220,6 +222,68 @@ See :pep:`692` for more details. (PEP written by Franek Magiera) +.. _whatsnew312-pep695: + +PEP 695: Type Parameter Syntax +------------------------------ + +Generic classes and functions under :pep:`484` were declared using a verbose syntax +that left the scope of type parameters unclear and required explicit declarations of +variance. + +:pep:`695` introduces a new, more compact and explicit way to create +:ref:`generic classes ` and :ref:`functions `:: + + def max[T](args: Iterable[T]) -> T: + ... + + class list[T]: + def __getitem__(self, index: int, /) -> T: + ... + + def append(self, element: T) -> None: + ... + +In addition, the PEP introduces a new way to write :ref:`type aliases ` +using the :keyword:`type` statement, which creates an instance of +:class:`~typing.TypeAliasType`:: + + type Point = tuple[float, float] + +Type aliases can also be :ref:`generic `:: + + type Point[T] = tuple[T, T] + +The new syntax also allows declaring :class:`~typing.TypeVarTuple` +and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.Typevar` +parameters with bounds or constraints:: + + type IntFunc[**P] = Callable[P, int] # ParamSpec + type LabeledTuple[*Ts] = tuple[str, *Ts] # TypeVarTuple + type HashableSequence[T: Hashable] = Sequence[T] # TypeVar with bound + type IntOrStrSequence[T: (int, str)] = Sequence[T] # TypeVar with constraints + +The value of type aliases and the bound and constraints of type variables +created through this syntax are evaluated only on demand, enabling the use of +forward references to types defined later in the file. + +Type parameters declared through a type parameter list are visible within the +scope of the declaration and any nested scopes, but not in the outer scope. For +example, they can be used in the type annotations for the methods of a generic +class or in the class body. However, they cannot be used in the module scope after +the class is defined. + +In order to support these scoping semantics, a new kind of scope is introduced, +the :ref:`annotation scope `. Annotation scopes behave for the +most part like function scopes, but interact differently with enclosing class scopes. +In Python 3.13, :term:`annotations ` will also be evaluated in +annotation scopes. + +See :pep:`695` for more details. + +(PEP written by Eric Traut. Implementation by Jelle Zijlstra, Eric Traut, +and others in :gh:`103764`.) + Other Language Changes ====================== From bc1c1d2367a14d9470cc68ba728479ba27c1bab6 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 19 May 2023 21:28:07 -0700 Subject: [PATCH 17/56] typo --- Doc/whatsnew/3.12.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 451850438cd954..c1f153c6758326 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -255,7 +255,7 @@ Type aliases can also be :ref:`generic `:: type Point[T] = tuple[T, T] The new syntax also allows declaring :class:`~typing.TypeVarTuple` -and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.Typevar` +and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.TypeVar` parameters with bounds or constraints:: type IntFunc[**P] = Callable[P, int] # ParamSpec From b1a843c4706cb7554ffb3e2504ccc0e7c3d1ea33 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sat, 20 May 2023 06:05:40 -0700 Subject: [PATCH 18/56] FQNs in ast.rst --- Doc/library/ast.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index cc94b2755567cb..67f5156002648e 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -1673,7 +1673,7 @@ aliases. .. class:: TypeVar(name, bound) - A :class:`~typing.TypeVar`. ``name`` is the name of the type variable, and + A :class:`typing.TypeVar`. ``name`` is the name of the type variable, and ``bound`` is the bound or constraints, if any. If the value is a :class:`Tuple`, it represents constraints; otherwise it represents the bound. @@ -1696,7 +1696,7 @@ aliases. .. class:: ParamSpec(name) - A :class:`~typing.ParamSpec`. ``name`` is the name of the parameter specification. + A :class:`typing.ParamSpec`. ``name`` is the name of the parameter specification. .. doctest:: @@ -1719,7 +1719,7 @@ aliases. .. class:: TypeVarTuple(name) - A :class:`~typing.TypeVarTuple`. ``name`` is the name of the type variable tuple. + A :class:`typing.TypeVarTuple`. ``name`` is the name of the type variable tuple. .. doctest:: From cc5b51c39521678e74e14e8a9920d4d936375c98 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 21 May 2023 06:00:30 -0700 Subject: [PATCH 19/56] Document TypeVar constructor signature, discuss variance --- Doc/library/typing.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e797e7164a951f..43f9f144b5fa5f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1361,7 +1361,8 @@ can also be created without the dedicated syntax, as documented below. ... # Etc. -.. class:: TypeVar +.. class:: TypeVar(name, *constraints, bound=None, covariant=False, + contravariant=False, infer_variance=False) Type variable. @@ -1453,7 +1454,10 @@ can also be created without the dedicated syntax, as documented below. At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, :func:`isinstance` and :func:`issubclass` should not be used with types. - Type variables may be marked covariant or contravariant by passing + The variance of type variables created through the :ref:`type parameter syntax ` + is inferred by type checkers. When ``infer_variance=True`` is passed, + variance is also inferred for manually created type variables. + Manually created type variables may be explicitly marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more details. By default, type variables are invariant. From b0e1ce30655869b8e270d5d74c58e5c830e16c69 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 21 May 2023 06:01:38 -0700 Subject: [PATCH 20/56] TypeVarTuple signature --- Doc/library/typing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 43f9f144b5fa5f..490f32668a051f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1466,7 +1466,7 @@ can also be created without the dedicated syntax, as documented below. Type variables can now be declared using the :ref:`type parameter ` syntax introduced by :pep:`695`. -.. class:: TypeVarTuple +.. class:: TypeVarTuple(name) Type variable tuple. A specialized form of :class:`type variable ` that enables *variadic* generics. From db94e4fa3cd9a5fb439bca362124451b9f8b017b Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 21 May 2023 06:48:06 -0700 Subject: [PATCH 21/56] Document TypeVar attributes, fix indentation --- Doc/library/typing.rst | 165 +++++++++++++++++++++++++--------------- Objects/typevarobject.c | 45 +++++------ 2 files changed, 123 insertions(+), 87 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 7626e0c6445ce5..55efd46baa686a 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1364,107 +1364,146 @@ can also be created without the dedicated syntax, as documented below. .. class:: TypeVar(name, *constraints, bound=None, covariant=False, contravariant=False, infer_variance=False) - Type variable. + Type variable. - Usage:: + Usage:: T = TypeVar('T') # Can be anything S = TypeVar('S', bound=str) # Can be any subtype of str A = TypeVar('A', str, bytes) # Must be exactly str or bytes - The syntax for :ref:`generic functions `, - :ref:`generic classes `, and - :ref:`generic type aliases ` can be used to - create type variables:: + The syntax for :ref:`generic functions `, + :ref:`generic classes `, and + :ref:`generic type aliases ` can be used to + create type variables:: - class Sequence[T]: # T is a TypeVar - ... + class Sequence[T]: # T is a TypeVar + ... - This syntax can also be used to create bound and constrained type - variables:: + This syntax can also be used to create bound and constrained type + variables:: - class StrSequence[S: str]: # S is a TypeVar bound to str - ... + class StrSequence[S: str]: # S is a TypeVar bound to str + ... - class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes - ... + class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes + ... + + Type variables exist primarily for the benefit of static type + checkers. They serve as the parameters for generic types as well + as for generic function definitions. See :class:`Generic` for more + information on generic types. Generic functions work as follows:: + + def repeat[T](x: T, n: int) -> Sequence[T]: + """Return a list containing n references to x.""" + return [x]*n + + + def print_capitalized[S: str](x: S) -> S: + """Print x capitalized, and return x.""" + print(x.capitalize()) + return x + + + def concatenate[A: (str, bytes)](x: A, y: A) -> A: + """Add two strings or bytes objects together.""" + return x + y - Type variables exist primarily for the benefit of static type - checkers. They serve as the parameters for generic types as well - as for generic function definitions. See :class:`Generic` for more - information on generic types. Generic functions work as follows:: + Note that type variables can be *bound*, *constrained*, or neither, but + cannot be both bound *and* constrained. - def repeat[T](x: T, n: int) -> Sequence[T]: - """Return a list containing n references to x.""" - return [x]*n + Bound type variables and constrained type variables have different + semantics in several important ways. Using a *bound* type variable means + that the ``TypeVar`` will be solved using the most specific type possible:: + x = print_capitalized('a string') + reveal_type(x) # revealed type is str - def print_capitalized[S: str](x: S) -> S: - """Print x capitalized, and return x.""" - print(x.capitalize()) - return x + class StringSubclass(str): + pass + + y = print_capitalized(StringSubclass('another string')) + reveal_type(y) # revealed type is StringSubclass + + z = print_capitalized(45) # error: int is not a subtype of str + + Type variables can be bound to concrete types, abstract types (ABCs or + protocols), and even unions of types:: + + U = TypeVar('U', bound=str|bytes) # Can be any subtype of the union str|bytes + V = TypeVar('V', bound=SupportsAbs) # Can be anything with an __abs__ method + + # Can be anything with an __abs__ method + def print_abs[V: SupportsAbs](arg: V) -> None: + print("Absolute value:", abs(arg)) + + .. _typing-constrained-typevar: + Using a *constrained* type variable, however, means that the ``TypeVar`` + can only ever be solved as being exactly one of the constraints given:: - def concatenate[A: (str, bytes)](x: A, y: A) -> A: - """Add two strings or bytes objects together.""" - return x + y + a = concatenate('one', 'two') + reveal_type(a) # revealed type is str + + b = concatenate(StringSubclass('one'), StringSubclass('two')) + reveal_type(b) # revealed type is str, despite StringSubclass being passed in + + c = concatenate('one', b'two') # error: type variable 'A' can be either str or bytes in a function call, but not both + + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issubclass` should not be used with types. + + The variance of type variables is inferred by type checkers when they are created + through the :ref:`type parameter syntax ` and when + ``infer_variance=True`` is passed. + Manually created type variables may be explicitly marked covariant or contravariant by passing + ``covariant=True`` or ``contravariant=True``. See :pep:`484` and :pep:`695` for more + details. By default, type variables are invariant. + + .. attribute:: __name__ - Note that type variables can be *bound*, *constrained*, or neither, but - cannot be both bound *and* constrained. + The name of the type variable. - Bound type variables and constrained type variables have different - semantics in several important ways. Using a *bound* type variable means - that the ``TypeVar`` will be solved using the most specific type possible:: + .. attribute:: __covariant__ - x = print_capitalized('a string') - reveal_type(x) # revealed type is str + Whether the type variable is covariant. - class StringSubclass(str): - pass + .. attribute:: __contravariant__ - y = print_capitalized(StringSubclass('another string')) - reveal_type(y) # revealed type is StringSubclass + Whether the type variable is contravariant. - z = print_capitalized(45) # error: int is not a subtype of str + .. attribute:: __infer_variance__ - Type variables can be bound to concrete types, abstract types (ABCs or - protocols), and even unions of types:: + Whether the type variable's variance should be inferred by type checkers. - U = TypeVar('U', bound=str|bytes) # Can be any subtype of the union str|bytes - V = TypeVar('V', bound=SupportsAbs) # Can be anything with an __abs__ method + .. versionadded:: 3.12 - # Can be anything with an __abs__ method - def print_abs[V: SupportsAbs](arg: V) -> None: - print("Absolute value:", abs(arg)) + .. attribute:: __bound__ -.. _typing-constrained-typevar: + The bound of the type variable, if any. - Using a *constrained* type variable, however, means that the ``TypeVar`` - can only ever be solved as being exactly one of the constraints given:: + .. versionchanged:: 3.12 - a = concatenate('one', 'two') - reveal_type(a) # revealed type is str + For type variables created through the new type parameter syntax, + the bound is evaluated only when the attribute is accessed, not when + the type variable is created. - b = concatenate(StringSubclass('one'), StringSubclass('two')) - reveal_type(b) # revealed type is str, despite StringSubclass being passed in + .. attribute:: __constraints__ - c = concatenate('one', b'two') # error: type variable 'A' can be either str or bytes in a function call, but not both + A tuple containing the constraints of the type variable, if any. - At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issubclass` should not be used with types. + .. versionchanged:: 3.12 - The variance of type variables created through the :ref:`type parameter syntax ` - is inferred by type checkers. When ``infer_variance=True`` is passed, - variance is also inferred for manually created type variables. - Manually created type variables may be explicitly marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default, type variables are invariant. + For type variables created through the new type parameter syntax, + the constraints are evaluated only when the attribute is accessed, not when + the type variable is created. .. versionchanged:: 3.12 Type variables can now be declared using the :ref:`type parameter ` syntax introduced by :pep:`695`. + The ``infer_variance`` parameter was added. .. class:: TypeVarTuple(name) diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 6730ebfc064e35..095ce3756305cc 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -446,42 +446,39 @@ PyDoc_STRVAR(typevar_doc, Usage::\n\ \n\ T = TypeVar('T') # Can be anything\n\ + S = TypeVar('S', bound=str) # Can be any subtype of str\n\ A = TypeVar('A', str, bytes) # Must be str or bytes\n\ \n\ +The syntax for generic functions, classes, and type aliases can\n\ +be used to create type variables::\n\ +\n\ + class Sequence[T]: # T is a TypeVar\n\ + ...\n\ +\n\ Type variables exist primarily for the benefit of static type\n\ checkers. They serve as the parameters for generic types as well\n\ as for generic function definitions. See class Generic for more\n\ information on generic types. Generic functions work as follows:\n\ \n\ - def repeat(x: T, n: int) -> List[T]:\n\ + def repeat[T](x: T, n: int) -> Sequence[T]:\n\ '''Return a list containing n references to x.'''\n\ return [x]*n\n\ \n\ - def longest(x: A, y: A) -> A:\n\ - '''Return the longest of two strings.'''\n\ - return x if len(x) >= len(y) else y\n\ -\n\ -The latter example's signature is essentially the overloading\n\ -of (str, str) -> str and (bytes, bytes) -> bytes. Also note\n\ -that if the arguments are instances of some subclass of str,\n\ -the return type is still plain str.\n\ -\n\ -At runtime, isinstance(x, T) and issubclass(C, T) will raise TypeError.\n\ -\n\ -Type variables defined with covariant=True or contravariant=True\n\ -can be used to declare covariant or contravariant generic types.\n\ -See PEP 484 for more details. By default generic types are invariant\n\ -in all type variables.\n\ -\n\ -Type variables can be introspected. e.g.:\n\ + def print_capitalized[S: str](x: S) -> S:\n\ + '''Print x capitalized, and return x.'''\n\ + print(x.capitalize())\n\ + return x\n\ \n\ - T.__name__ == 'T'\n\ - T.__constraints__ == ()\n\ - T.__covariant__ == False\n\ - T.__contravariant__ = False\n\ - A.__constraints__ == (str, bytes)\n\ + def concatenate[A: (str, bytes)](x: A, y: A) -> A:\n\ + '''Add two strings or bytes objects together.'''\n\ + return x + y\n\ \n\ -Note that only type variables defined in global scope can be pickled.\n\ +The variance of type variables is inferred by type checkers when they are created\n\ +through the type parameter syntax and when\n\ +``infer_variance=True`` is passed.\n\ +Manually created type variables may be explicitly marked covariant or contravariant by passing\n\ +``covariant=True`` or ``contravariant=True``. See PEP 484 and PEP 695 for more\n\ +details. By default, type variables are invariant.\n\ "); static PyType_Slot typevar_slots[] = { From 18b9d63673b2af519aa01f7fec16167d82216ee3 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 21 May 2023 06:50:21 -0700 Subject: [PATCH 22/56] Fix TypeVarTuple indentation --- Doc/library/typing.rst | 158 ++++++++++++++++++++++------------------- 1 file changed, 83 insertions(+), 75 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 55efd46baa686a..04b729db0108cf 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1507,108 +1507,112 @@ can also be created without the dedicated syntax, as documented below. .. class:: TypeVarTuple(name) - Type variable tuple. A specialized form of :class:`type variable ` - that enables *variadic* generics. + Type variable tuple. A specialized form of :class:`type variable ` + that enables *variadic* generics. - Type variable tuples can be declared in :ref:`type parameter lists ` - using a single asterisk (``*``) before the name:: + Type variable tuples can be declared in :ref:`type parameter lists ` + using a single asterisk (``*``) before the name:: - def move_first_element_to_last[T, *Ts](tup: tuple[T, *Ts]) -> tuple[*Ts, T]: - return (*tup[1:], tup[0]) + def move_first_element_to_last[T, *Ts](tup: tuple[T, *Ts]) -> tuple[*Ts, T]: + return (*tup[1:], tup[0]) - Or by explicitly invoking the ``TypeVarTuple`` constructor:: + Or by explicitly invoking the ``TypeVarTuple`` constructor:: - T = TypeVar("T") - Ts = TypeVarTuple("Ts") + T = TypeVar("T") + Ts = TypeVarTuple("Ts") - def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]: - return (*tup[1:], tup[0]) + def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]: + return (*tup[1:], tup[0]) - A normal type variable enables parameterization with a single type. A type - variable tuple, in contrast, allows parameterization with an - *arbitrary* number of types by acting like an *arbitrary* number of type - variables wrapped in a tuple. For example:: + A normal type variable enables parameterization with a single type. A type + variable tuple, in contrast, allows parameterization with an + *arbitrary* number of types by acting like an *arbitrary* number of type + variables wrapped in a tuple. For example:: - # T is bound to int, Ts is bound to () - # Return value is (1,), which has type tuple[int] - move_first_element_to_last(tup=(1,)) + # T is bound to int, Ts is bound to () + # Return value is (1,), which has type tuple[int] + move_first_element_to_last(tup=(1,)) - # T is bound to int, Ts is bound to (str,) - # Return value is ('spam', 1), which has type tuple[str, int] - move_first_element_to_last(tup=(1, 'spam')) + # T is bound to int, Ts is bound to (str,) + # Return value is ('spam', 1), which has type tuple[str, int] + move_first_element_to_last(tup=(1, 'spam')) - # T is bound to int, Ts is bound to (str, float) - # Return value is ('spam', 3.0, 1), which has type tuple[str, float, int] - move_first_element_to_last(tup=(1, 'spam', 3.0)) + # T is bound to int, Ts is bound to (str, float) + # Return value is ('spam', 3.0, 1), which has type tuple[str, float, int] + move_first_element_to_last(tup=(1, 'spam', 3.0)) - # This fails to type check (and fails at runtime) - # because tuple[()] is not compatible with tuple[T, *Ts] - # (at least one element is required) - move_first_element_to_last(tup=()) + # This fails to type check (and fails at runtime) + # because tuple[()] is not compatible with tuple[T, *Ts] + # (at least one element is required) + move_first_element_to_last(tup=()) - Note the use of the unpacking operator ``*`` in ``tuple[T, *Ts]``. - Conceptually, you can think of ``Ts`` as a tuple of type variables - ``(T1, T2, ...)``. ``tuple[T, *Ts]`` would then become - ``tuple[T, *(T1, T2, ...)]``, which is equivalent to - ``tuple[T, T1, T2, ...]``. (Note that in older versions of Python, you might - see this written using :data:`Unpack ` instead, as - ``Unpack[Ts]``.) + Note the use of the unpacking operator ``*`` in ``tuple[T, *Ts]``. + Conceptually, you can think of ``Ts`` as a tuple of type variables + ``(T1, T2, ...)``. ``tuple[T, *Ts]`` would then become + ``tuple[T, *(T1, T2, ...)]``, which is equivalent to + ``tuple[T, T1, T2, ...]``. (Note that in older versions of Python, you might + see this written using :data:`Unpack ` instead, as + ``Unpack[Ts]``.) - Type variable tuples must *always* be unpacked. This helps distinguish type - variable tuples from normal type variables:: + Type variable tuples must *always* be unpacked. This helps distinguish type + variable tuples from normal type variables:: - x: Ts # Not valid - x: tuple[Ts] # Not valid - x: tuple[*Ts] # The correct way to do it + x: Ts # Not valid + x: tuple[Ts] # Not valid + x: tuple[*Ts] # The correct way to do it - Type variable tuples can be used in the same contexts as normal type - variables. For example, in class definitions, arguments, and return types:: + Type variable tuples can be used in the same contexts as normal type + variables. For example, in class definitions, arguments, and return types:: - class Array[*Shape]: - def __getitem__(self, key: tuple[*Shape]) -> float: ... - def __abs__(self) -> "Array[*Shape]": ... - def get_shape(self) -> tuple[*Shape]: ... + class Array[*Shape]: + def __getitem__(self, key: tuple[*Shape]) -> float: ... + def __abs__(self) -> "Array[*Shape]": ... + def get_shape(self) -> tuple[*Shape]: ... - Type variable tuples can be happily combined with normal type variables:: + Type variable tuples can be happily combined with normal type variables:: - DType = TypeVar('DType') + DType = TypeVar('DType') - class Array[DType, *Shape]: # This is fine - pass + class Array[DType, *Shape]: # This is fine + pass - class Array2[*Shape, DType]: # This would also be fine - pass + class Array2[*Shape, DType]: # This would also be fine + pass - float_array_1d: Array[float, Height] = Array() # Totally fine - int_array_2d: Array[int, Height, Width] = Array() # Yup, fine too + float_array_1d: Array[float, Height] = Array() # Totally fine + int_array_2d: Array[int, Height, Width] = Array() # Yup, fine too - However, note that at most one type variable tuple may appear in a single - list of type arguments or type parameters:: + However, note that at most one type variable tuple may appear in a single + list of type arguments or type parameters:: - x: tuple[*Ts, *Ts] # Not valid - class Array[*Shape, *Shape]: # Not valid - pass + x: tuple[*Ts, *Ts] # Not valid + class Array[*Shape, *Shape]: # Not valid + pass - Finally, an unpacked type variable tuple can be used as the type annotation - of ``*args``:: + Finally, an unpacked type variable tuple can be used as the type annotation + of ``*args``:: - def call_soon[*Ts]( - callback: Callable[[*Ts], None], - *args: *Ts - ) -> None: - ... - callback(*args) + def call_soon[*Ts]( + callback: Callable[[*Ts], None], + *args: *Ts + ) -> None: + ... + callback(*args) - In contrast to non-unpacked annotations of ``*args`` - e.g. ``*args: int``, - which would specify that *all* arguments are ``int`` - ``*args: *Ts`` - enables reference to the types of the *individual* arguments in ``*args``. - Here, this allows us to ensure the types of the ``*args`` passed - to ``call_soon`` match the types of the (positional) arguments of - ``callback``. + In contrast to non-unpacked annotations of ``*args`` - e.g. ``*args: int``, + which would specify that *all* arguments are ``int`` - ``*args: *Ts`` + enables reference to the types of the *individual* arguments in ``*args``. + Here, this allows us to ensure the types of the ``*args`` passed + to ``call_soon`` match the types of the (positional) arguments of + ``callback``. - See :pep:`646` for more details on type variable tuples. + See :pep:`646` for more details on type variable tuples. - .. versionadded:: 3.11 + .. attribute:: __name__ + + The name of the type variable tuple. + + .. versionadded:: 3.11 .. versionchanged:: 3.12 @@ -1686,6 +1690,10 @@ can also be created without the dedicated syntax, as documented below. :class:`TypeVar`. However the actual semantics of these keywords are yet to be decided. + .. attribute:: __name__ + + The name of the parameter specification. + .. versionadded:: 3.10 .. versionchanged:: 3.12 From fd1d0a8e471d92aa148e977ce0f45efd0a0cf9b3 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 05:57:38 -0700 Subject: [PATCH 23/56] s/typeparams/type_params/ --- Doc/library/ast.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 75ae581e7fbed6..a240e597068b14 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -917,10 +917,10 @@ Statements type_ignores=[]) -.. class:: TypeAlias(name, typeparams, value) +.. class:: TypeAlias(name, type_params, value) A type alias created through the :keyword:`type` statement. ``name`` - is the name of the alias, ``typeparams`` is a list of + is the name of the alias, ``type_params`` is a list of :ref:`type parameters `, and ``value`` is the value of the type alias. @@ -931,7 +931,7 @@ Statements body=[ TypeAlias( name=Name(id='Alias', ctx=Store()), - typeparams=[], + type_params=[], value=Name(id='int', ctx=Load()))], type_ignores=[]) @@ -1684,7 +1684,7 @@ aliases. body=[ TypeAlias( name=Name(id='Alias', ctx=Store()), - typeparams=[ + type_params=[ TypeVar( name='T', bound=Name(id='int', ctx=Load()))], @@ -1705,7 +1705,7 @@ aliases. body=[ TypeAlias( name=Name(id='Alias', ctx=Store()), - typeparams=[ + type_params=[ ParamSpec(name='P')], value=Subscript( value=Name(id='Callable', ctx=Load()), @@ -1728,7 +1728,7 @@ aliases. body=[ TypeAlias( name=Name(id='Alias', ctx=Store()), - typeparams=[ + type_params=[ TypeVarTuple(name='Ts')], value=Subscript( value=Name(id='tuple', ctx=Load()), @@ -1744,12 +1744,12 @@ aliases. Function and class definitions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. class:: FunctionDef(name, typeparams, args, body, decorator_list, returns, type_comment) +.. class:: FunctionDef(name, type_params, args, body, decorator_list, returns, type_comment) A function definition. * ``name`` is a raw string of the function name. - * ``typeparams`` is a list of :ref:`type parameters `. + * ``type_params`` is a list of :ref:`type parameters `. * ``args`` is an :class:`arguments` node. * ``body`` is the list of nodes inside the function. * ``decorator_list`` is the list of decorators to be applied, stored outermost @@ -1917,12 +1917,12 @@ Function and class definitions type_ignores=[]) -.. class:: ClassDef(name, typeparams, bases, keywords, starargs, kwargs, body, decorator_list) +.. class:: ClassDef(name, type_params, bases, keywords, starargs, kwargs, body, decorator_list) A class definition. * ``name`` is a raw string for the class name - * ``typeparams`` is a list of :ref:`type parameters `. + * ``type_params`` is a list of :ref:`type parameters `. * ``bases`` is a list of nodes for explicitly specified base classes. * ``keywords`` is a list of :class:`keyword` nodes, principally for 'metaclass'. Other keywords will be passed to the metaclass, as per `PEP-3115 From 62ba887cee69340fd5fae8c9594f6b3cb54bb0dd Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 06:15:22 -0700 Subject: [PATCH 24/56] Apply suggestions from code review Thanks Alex! Co-authored-by: Alex Waygood --- Doc/library/ast.rst | 4 ++-- Doc/library/dis.rst | 4 ++-- Doc/library/typing.rst | 20 ++++++++++---------- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index a240e597068b14..173dc9d712ac5c 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -1673,8 +1673,8 @@ aliases. .. class:: TypeVar(name, bound) - A :class:`typing.TypeVar`. ``name`` is the name of the type variable, and - ``bound`` is the bound or constraints, if any. If the value is a :class:`Tuple`, + A :class:`typing.TypeVar`. ``name`` is the name of the type variable. + ``bound`` is the bound or constraints, if any. If ``bound`` is a :class:`Tuple`, it represents constraints; otherwise it represents the bound. .. doctest:: diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 2095dc4986935a..9be6b1c100e6c9 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -941,7 +941,7 @@ iterations of the loop. .. opcode:: LOAD_FROM_DICT_OR_GLOBALS (i) Pops a mapping off the stack and looks up the value for ``co_names[namei]``. - If the name is not found there, look it up in the globals and then the builtins, + If the name is not found there, looks it up in the globals and then the builtins, similar to :opcode:`LOAD_GLOBAL`. This is used for loading global variables in :ref:`annotation scopes ` within class bodies. @@ -1268,7 +1268,7 @@ iterations of the loop. Pops a mapping off the stack and looks up the name associated with slot ``i`` of the "fast locals" storage in this mapping. - If the name is not found there, load it from the cell contained in + If the name is not found there, loads it from the cell contained in slot ``i``, similar to :opcode:`LOAD_DEREF`. This is used for loading free variables in class bodies and in :ref:`annotation scopes ` within class bodies. diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 04b729db0108cf..40c08a429fbce3 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -344,13 +344,13 @@ A user-defined class can be defined as a generic class. def log(self, message: str) -> None: self.logger.info('%s: %s', self.name, message) -This syntax indicates that the class ``LoggedVar`` takes a -single type parameter ``T`` . This also makes ``T`` valid as a type within the -class body. +This syntax indicates that the class ``LoggedVar`` is parameterised around a +single :class:`type variable ` ``T`` . This also makes ``T`` valid as +a type within the class body. Generic classes implicitly inherit from :class:`Generic`. For compatibility with Python 3.11 and lower, it is also possible to inherit explicitly from -:class:`Generic`:: +:class:`Generic` to indicate a generic class:: from typing import TypeVar, Generic @@ -359,8 +359,8 @@ with Python 3.11 and lower, it is also possible to inherit explicitly from class LoggedVar(Generic[T]): ... -The :class:`Generic` base class defines :meth:`~object.__class_getitem__` so -that ``LoggedVar[T]`` is valid as a type:: +Generic classes have :meth:`~object.__class_getitem__` methods, meaning they +can be parameterised at runtime (e.g. ``LoggedVar[int]`` below):: from collections.abc import Iterable @@ -404,7 +404,7 @@ Generic classes can also inherit from other classes:: class LinkedList[T](Sized): ... -When inheriting from generic classes, some type variables could be fixed:: +When inheriting from generic classes, some type parameters could be fixed:: from collections.abc import Mapping @@ -421,7 +421,7 @@ not generic but implicitly inherits from ``Iterable[Any]``:: class MyIterable(Iterable): # Same as Iterable[Any] -User defined generic type aliases are also supported. Examples:: +User-defined generic type aliases are also supported. Examples:: from collections.abc import Iterable @@ -450,7 +450,7 @@ through a simple assignment:: .. versionchanged:: 3.12 Syntactic support for generics and type aliases is new in version 3.12. - Previously, generic classes had to inherit from :class:`Generic` + Previously, generic classes had to explicitly inherit from :class:`Generic` or contain a type variable in one of their bases. User-defined generics for parameter expressions are also supported via parameter @@ -459,7 +459,7 @@ with type variables' described above as parameter specification variables are treated by the typing module as a specialized type variable. The one exception to this is that a list of types can be used to substitute a :class:`ParamSpec`:: - >>> class Z[T, **P]: ... # P is a ParamSpec + >>> class Z[T, **P]: ... # T is a TypeVar; P is a ParamSpec ... >>> Z[int, [dict, float]] __main__.Z[int, [dict, float]] From 200eba1f5ff1c5a63a1db876343ea5b32ab0e312 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 06:00:58 -0700 Subject: [PATCH 25/56] Reword long dis sentence --- Doc/library/dis.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 9be6b1c100e6c9..c135ebd70a73e3 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -188,9 +188,9 @@ operation is being performed, so the intermediate analysis object isn't useful: For a module, it disassembles all functions. For a class, it disassembles all methods (including class and static methods). For a code object or sequence of raw bytecode, it prints one line per bytecode instruction. - It also recursively disassembles nested code objects (the code of - generator expressions and nested functions, and the code - used for building nested classes and :ref:`annotation scopes `). + It also recursively disassembles nested code objects. These can include + generator expressions, nested functions, the bodies of nested classes, + and the code objects used for :ref:`annotation scopes `. Strings are first compiled to code objects with the :func:`compile` built-in function before being disassembled. If no object is provided, this function disassembles the last traceback. From adaf87431496c98e4d7e8b78e509938ca6a3bffc Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 06:15:34 -0700 Subject: [PATCH 26/56] Manual changes from Alex review --- Doc/library/typing.rst | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 40c08a429fbce3..47710c72bd5828 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -102,7 +102,7 @@ annotations. These include: *Introducing* a new way of typing ``**kwargs`` with :data:`Unpack` and :data:`TypedDict` * :pep:`695`: Type Parameter Syntax - *Introducing* syntax for creating generic functions, classes, and type aliases. + *Introducing* builtin syntax for creating generic functions, classes, and type aliases. * :pep:`698`: Adding an override decorator to typing *Introducing* the :func:`@override` decorator @@ -113,7 +113,8 @@ Type aliases A type alias is defined using the :keyword:`type` statement, which creates an instance of :class:`TypeAliasType`. In this example, -``Vector`` and ``list[float]`` will be treated as interchangeable synonyms:: +``Vector`` and ``list[float]`` will be treated equivalently by static type +checkers:: type Vector = list[float] @@ -149,7 +150,8 @@ compatibility, type aliases can also be created through simple assignment:: Vector = list[float] -Or marked with :data:`TypeAlias`:: +Or marked with :data:`TypeAlias` to make it explicit that this is a type alias, +not a normal variable assignment:: from typing import TypeAlias @@ -220,7 +222,7 @@ See :pep:`484` for more details. .. note:: Recall that the use of a type alias declares two types to be *equivalent* to - one another. Doing ``Alias = Original`` will make the static type checker + one another. Doing ``type Alias = Original`` will make the static type checker treat ``Alias`` as being *exactly equivalent* to ``Original`` in all cases. This is useful when you want to simplify complex type signatures. @@ -308,9 +310,9 @@ Or by using the :class:`TypeVar` factory directly:: from collections.abc import Sequence from typing import TypeVar - T = TypeVar('T') # Declare type variable + U = TypeVar('U') # Declare type variable - def first(l: Sequence[T]) -> T: # Generic function + def first(l: Sequence[U]) -> U: # Generic function return l[0] .. versionchanged:: 3.12 @@ -373,14 +375,14 @@ A generic type can have any number of type variables. All varieties of from typing import TypeVar, Generic, Sequence - T = TypeVar('T', contravariant=True) - B = TypeVar('B', bound=Sequence[bytes], covariant=True) - S = TypeVar('S', int, str) - - class OldWeirdTrio(Generic[T, B, S]): + class NewWeirdTrio[T, B: Sequence[bytes], S: (int, str)]: ... - class NewWeirdTrio[T, B: Sequence[bytes], S: (int, str)]: + OldT = TypeVar('OldT', contravariant=True) + OldB = TypeVar('OldB', bound=Sequence[bytes], covariant=True) + OldS = TypeVar('OldS', int, str) + + class OldWeirdTrio(Generic[OldT, OldB, OldS]): ... Each type variable argument to :class:`Generic` must be distinct. @@ -389,7 +391,7 @@ This is thus invalid:: from typing import TypeVar, Generic ... - class Pair[T, T]: # SyntaxError + class Pair[M, M]: # SyntaxError ... T = TypeVar('T') @@ -474,7 +476,7 @@ inheritance from :class:`Generic`. In this case, ``**`` is not used:: class Z(Generic[P]): ... -Furthermore, a generic with only one parameter specification variable will accept +A generic with only one parameter specification variable will accept parameter lists in the forms ``X[[Type1, Type2, ...]]`` and also ``X[Type1, Type2, ...]`` for aesthetic reasons. Internally, the latter is converted to the former, so the following are equivalent:: @@ -818,6 +820,8 @@ These can be used as types in annotations and do not support ``[]``. Note that while :data:`TypeAlias` and :class:`TypeAliasType` serve similar purposes and have similar names, they are distinct and the latter is not the type of the former. + Removal of :data:`TypeAlias` is not currently planned, but users + are encouraged to migrate to :keyword:`type` statements. Special forms """"""""""""" From 365ae7bec57f13db76fc8fc908f47696691997fd Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 06:18:41 -0700 Subject: [PATCH 27/56] Improve ast.TypeAlias text --- Doc/library/ast.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 173dc9d712ac5c..0efe619a09ec3f 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -919,8 +919,8 @@ Statements .. class:: TypeAlias(name, type_params, value) - A type alias created through the :keyword:`type` statement. ``name`` - is the name of the alias, ``type_params`` is a list of + A :ref:`type alias ` created through the :keyword:`type` + statement. ``name`` is the name of the alias, ``type_params`` is a list of :ref:`type parameters `, and ``value`` is the value of the type alias. From 96e5d1652c222aaaa91f658f30fcd576f0e1ba2a Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 07:25:33 -0700 Subject: [PATCH 28/56] Document __type_params__ --- Doc/library/stdtypes.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 9203afbf6a4e8a..fdef5314b9a4ef 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -5476,6 +5476,14 @@ types, where they are relevant. Some of these are not reported by the .. versionadded:: 3.3 +.. attribute:: definition.__type_params__ + + The :ref:`type parameters ` of generic classes, functions, + and :ref:`type aliases `. + + .. versionadded:: 3.12 + + .. attribute:: class.__mro__ This attribute is a tuple of classes that are considered when looking for From 0204d540e6d933688e12b2d0cab361bc8385e7a1 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 08:11:33 -0700 Subject: [PATCH 29/56] Add to datamodel.rst --- Doc/reference/datamodel.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index c0734e49f29192..e8f9775dd33ce1 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -499,6 +499,7 @@ Callable types single: __globals__ (function attribute) single: __annotations__ (function attribute) single: __kwdefaults__ (function attribute) + single: __type_params__ (function attribute) pair: global; namespace +-------------------------+-------------------------------+-----------+ @@ -561,6 +562,12 @@ Callable types | :attr:`__kwdefaults__` | A dict containing defaults | Writable | | | for keyword-only parameters. | | +-------------------------+-------------------------------+-----------+ + | :attr:`__type_params__` | A tuple containing the | Writable | + | | :ref:`type parameters | | + | | ` of a | | + | | :ref:`generic function | | + | | `. | | + +-------------------------+-------------------------------+-----------+ Most of the attributes labelled "Writable" check the type of the assigned value. @@ -837,6 +844,7 @@ Custom classes single: __bases__ (class attribute) single: __doc__ (class attribute) single: __annotations__ (class attribute) + single: __type_params__ (class attribute) Special attributes: @@ -863,6 +871,10 @@ Custom classes working with :attr:`__annotations__`, please see :ref:`annotations-howto`. + :attr:`__type_params__` + A tuple containing the :ref:`type parameters ` of + a :ref:`generic class `. + Class instances .. index:: pair: object; class instance From f426d394979df18ddefe0a056c0768ca43687c5e Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 08:17:26 -0700 Subject: [PATCH 30/56] Wording change --- Doc/library/typing.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 47710c72bd5828..82304003781f2b 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1325,7 +1325,8 @@ These can be used as types in annotations using ``[]``, each having a unique syn Building generic types and type aliases """"""""""""""""""""""""""""""""""""""" -These are not used in annotations. They are building blocks for creating generic types. +The following objects are not used directly in annotations. Instead, they are building blocks +for creating generic types and type aliases. They can be created through special syntax (:ref:`type parameter lists ` and the :keyword:`type` statement). For compatibility with Python 3.11 and earlier, they can also be created without the dedicated syntax, as documented below. From 60d7783b47004e584e4c6a3ac5a4d9462b596dcc Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 08:19:13 -0700 Subject: [PATCH 31/56] Update Doc/library/typing.rst Co-authored-by: Alex Waygood --- Doc/library/typing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 82304003781f2b..91ed4620850463 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -476,7 +476,7 @@ inheritance from :class:`Generic`. In this case, ``**`` is not used:: class Z(Generic[P]): ... -A generic with only one parameter specification variable will accept +Another difference between :class:`TypeVar` and :class:`ParamSpec` is that a generic with only one parameter specification variable will accept parameter lists in the forms ``X[[Type1, Type2, ...]]`` and also ``X[Type1, Type2, ...]`` for aesthetic reasons. Internally, the latter is converted to the former, so the following are equivalent:: From 63d4d842c3603e3267e6a93a8d49b3a193eeeeeb Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 09:06:59 -0700 Subject: [PATCH 32/56] Apply suggestions from code review Co-authored-by: Alex Waygood --- Doc/library/typing.rst | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 91ed4620850463..1946527da8a7f3 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -375,7 +375,7 @@ A generic type can have any number of type variables. All varieties of from typing import TypeVar, Generic, Sequence - class NewWeirdTrio[T, B: Sequence[bytes], S: (int, str)]: + class WeirdTrio[T, B: Sequence[bytes], S: (int, str)]: ... OldT = TypeVar('OldT', contravariant=True) @@ -476,7 +476,8 @@ inheritance from :class:`Generic`. In this case, ``**`` is not used:: class Z(Generic[P]): ... -Another difference between :class:`TypeVar` and :class:`ParamSpec` is that a generic with only one parameter specification variable will accept +Another difference between :class:`TypeVar` and :class:`ParamSpec` is that a +generic with only one parameter specification variable will accept parameter lists in the forms ``X[[Type1, Type2, ...]]`` and also ``X[Type1, Type2, ...]`` for aesthetic reasons. Internally, the latter is converted to the former, so the following are equivalent:: @@ -1327,9 +1328,11 @@ Building generic types and type aliases The following objects are not used directly in annotations. Instead, they are building blocks for creating generic types and type aliases. -They can be created through special syntax (:ref:`type parameter lists ` -and the :keyword:`type` statement). For compatibility with Python 3.11 and earlier, they -can also be created without the dedicated syntax, as documented below. + +These objects can be created through special syntax +(:ref:`type parameter lists ` and the :keyword:`type` statement). +For compatibility with Python 3.11 and earlier, they can also be created +without the dedicated syntax, as documented below. .. class:: Generic @@ -1356,7 +1359,7 @@ can also be created without the dedicated syntax, as documented below. return default Generic classes can also be declared by explicitly inheriting from - ``Generic``. In this case, the type parameters must be created + ``Generic``. In this case, the type parameters must be declared separately:: KT = TypeVar('KT') @@ -1401,14 +1404,14 @@ can also be created without the dedicated syntax, as documented below. information on generic types. Generic functions work as follows:: def repeat[T](x: T, n: int) -> Sequence[T]: - """Return a list containing n references to x.""" - return [x]*n + """Return a list containing n references to x.""" + return [x]*n def print_capitalized[S: str](x: S) -> S: - """Print x capitalized, and return x.""" - print(x.capitalize()) - return x + """Print x capitalized, and return x.""" + print(x.capitalize()) + return x def concatenate[A: (str, bytes)](x: A, y: A) -> A: From fd6751a2ad597ef060e9628730543db3fb7dbbb9 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 09:10:54 -0700 Subject: [PATCH 33/56] fix indentation --- Doc/library/typing.rst | 84 +++++++++++++++++++++--------------------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 1946527da8a7f3..56856ff7d9683a 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -718,20 +718,20 @@ These can be used as types in annotations and do not support ``[]``. This can be used to define a function that should never be called, or a function that never returns:: - from typing import Never + from typing import Never - def never_call_me(arg: Never) -> None: - pass + def never_call_me(arg: Never) -> None: + pass - def int_or_str(arg: int | str) -> None: - never_call_me(arg) # type checker error - match arg: - case int(): - print("It's an int") - case str(): - print("It's a str") - case _: - never_call_me(arg) # ok, arg is of type Never + def int_or_str(arg: int | str) -> None: + never_call_me(arg) # type checker error + match arg: + case int(): + print("It's an int") + case str(): + print("It's a str") + case _: + never_call_me(arg) # ok, arg is of type Never .. versionadded:: 3.11 @@ -765,9 +765,9 @@ These can be used as types in annotations and do not support ``[]``. from typing import Self class Foo: - def return_self(self) -> Self: - ... - return self + def return_self(self) -> Self: + ... + return self This annotation is semantically equivalent to the following, @@ -778,16 +778,16 @@ These can be used as types in annotations and do not support ``[]``. Self = TypeVar("Self", bound="Foo") class Foo: - def return_self(self: Self) -> Self: - ... - return self + def return_self(self: Self) -> Self: + ... + return self In general if something currently follows the pattern of:: class Foo: - def return_self(self) -> "Foo": - ... - return self + def return_self(self) -> "Foo": + ... + return self You should use :data:`Self` as calls to ``SubclassOfFoo.return_self`` would have ``Foo`` as the return type and not ``SubclassOfFoo``. @@ -1312,8 +1312,8 @@ These can be used as types in annotations using ``[]``, each having a unique syn from typing import TypedDict, Unpack class Movie(TypedDict): - name: str - year: int + name: str + year: int # This function expects two keyword arguments - `name` of type `str` # and `year` of type `int`. @@ -1386,17 +1386,17 @@ without the dedicated syntax, as documented below. create type variables:: class Sequence[T]: # T is a TypeVar - ... + ... This syntax can also be used to create bound and constrained type variables:: class StrSequence[S: str]: # S is a TypeVar bound to str - ... + ... class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes - ... + ... Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well @@ -1415,8 +1415,8 @@ without the dedicated syntax, as documented below. def concatenate[A: (str, bytes)](x: A, y: A) -> A: - """Add two strings or bytes objects together.""" - return x + y + """Add two strings or bytes objects together.""" + return x + y Note that type variables can be *bound*, *constrained*, or neither, but cannot be both bound *and* constrained. @@ -1429,7 +1429,7 @@ without the dedicated syntax, as documented below. reveal_type(x) # revealed type is str class StringSubclass(str): - pass + pass y = print_capitalized(StringSubclass('another string')) reveal_type(y) # revealed type is StringSubclass @@ -1444,7 +1444,7 @@ without the dedicated syntax, as documented below. # Can be anything with an __abs__ method def print_abs[V: SupportsAbs](arg: V) -> None: - print("Absolute value:", abs(arg)) + print("Absolute value:", abs(arg)) .. _typing-constrained-typevar: @@ -1522,7 +1522,7 @@ without the dedicated syntax, as documented below. using a single asterisk (``*``) before the name:: def move_first_element_to_last[T, *Ts](tup: tuple[T, *Ts]) -> tuple[*Ts, T]: - return (*tup[1:], tup[0]) + return (*tup[1:], tup[0]) Or by explicitly invoking the ``TypeVarTuple`` constructor:: @@ -1530,7 +1530,7 @@ without the dedicated syntax, as documented below. Ts = TypeVarTuple("Ts") def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]: - return (*tup[1:], tup[0]) + return (*tup[1:], tup[0]) A normal type variable enables parameterization with a single type. A type variable tuple, in contrast, allows parameterization with an @@ -1573,19 +1573,19 @@ without the dedicated syntax, as documented below. variables. For example, in class definitions, arguments, and return types:: class Array[*Shape]: - def __getitem__(self, key: tuple[*Shape]) -> float: ... - def __abs__(self) -> "Array[*Shape]": ... - def get_shape(self) -> tuple[*Shape]: ... + def __getitem__(self, key: tuple[*Shape]) -> float: ... + def __abs__(self) -> "Array[*Shape]": ... + def get_shape(self) -> tuple[*Shape]: ... Type variable tuples can be happily combined with normal type variables:: DType = TypeVar('DType') class Array[DType, *Shape]: # This is fine - pass + pass class Array2[*Shape, DType]: # This would also be fine - pass + pass float_array_1d: Array[float, Height] = Array() # Totally fine int_array_2d: Array[int, Height, Width] = Array() # Yup, fine too @@ -1595,7 +1595,7 @@ without the dedicated syntax, as documented below. x: tuple[*Ts, *Ts] # Not valid class Array[*Shape, *Shape]: # Not valid - pass + pass Finally, an unpacked type variable tuple can be used as the type annotation of ``*args``:: @@ -1604,8 +1604,8 @@ without the dedicated syntax, as documented below. callback: Callable[[*Ts], None], *args: *Ts ) -> None: - ... - callback(*args) + ... + callback(*args) In contrast to non-unpacked annotations of ``*args`` - e.g. ``*args: int``, which would specify that *all* arguments are ``int`` - ``*args: *Ts`` @@ -2428,8 +2428,8 @@ Corresponding to collections in :mod:`collections.abc` A generic version of :class:`collections.abc.Mapping`. This type can be used as follows:: - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] .. deprecated:: 3.9 :class:`collections.abc.Mapping` now supports subscripting (``[]``). From 499dae1518ed5e8ef37c97f3f7e2d896efefc325 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 09:12:42 -0700 Subject: [PATCH 34/56] Link to generic functions --- Doc/library/typing.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 56856ff7d9683a..eca067fdc6387e 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1358,6 +1358,9 @@ without the dedicated syntax, as documented below. except KeyError: return default + Here the brackets after the function name indicate a + :ref:`generic function `. + Generic classes can also be declared by explicitly inheriting from ``Generic``. In this case, the type parameters must be declared separately:: From b4172c99cd36dfc46af80fbec57710a9a3000baf Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 09:14:05 -0700 Subject: [PATCH 35/56] Move up variance paragraph --- Doc/library/typing.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index eca067fdc6387e..1d0db5164492a3 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1424,6 +1424,13 @@ without the dedicated syntax, as documented below. Note that type variables can be *bound*, *constrained*, or neither, but cannot be both bound *and* constrained. + The variance of type variables is inferred by type checkers when they are created + through the :ref:`type parameter syntax ` and when + ``infer_variance=True`` is passed. + Manually created type variables may be explicitly marked covariant or contravariant by passing + ``covariant=True`` or ``contravariant=True``. See :pep:`484` and :pep:`695` for more + details. By default, type variables are invariant. + Bound type variables and constrained type variables have different semantics in several important ways. Using a *bound* type variable means that the ``TypeVar`` will be solved using the most specific type possible:: @@ -1465,13 +1472,6 @@ without the dedicated syntax, as documented below. At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, :func:`isinstance` and :func:`issubclass` should not be used with types. - The variance of type variables is inferred by type checkers when they are created - through the :ref:`type parameter syntax ` and when - ``infer_variance=True`` is passed. - Manually created type variables may be explicitly marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` and :pep:`695` for more - details. By default, type variables are invariant. - .. attribute:: __name__ The name of the type variable. From 0d37675705e0e6a2e32a94131d49f18777016eac Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 18:48:14 -0700 Subject: [PATCH 36/56] Some Alex comments --- Doc/library/typing.rst | 41 ++++++++++++++++----------------- Objects/typevarobject.c | 50 ++++++++++++++++++++--------------------- 2 files changed, 46 insertions(+), 45 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 1d0db5164492a3..74a9c03a1d4b3f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1377,16 +1377,10 @@ without the dedicated syntax, as documented below. Type variable. - Usage:: - - T = TypeVar('T') # Can be anything - S = TypeVar('S', bound=str) # Can be any subtype of str - A = TypeVar('A', str, bytes) # Must be exactly str or bytes - - The syntax for :ref:`generic functions `, + The preferred way to construct a type variable is via the dedicated syntax + for :ref:`generic functions `, :ref:`generic classes `, and - :ref:`generic type aliases ` can be used to - create type variables:: + :ref:`generic type aliases `:: class Sequence[T]: # T is a TypeVar ... @@ -1401,9 +1395,16 @@ without the dedicated syntax, as documented below. class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes ... + However, if desired, reusable type variables can also be constructed manually, like so:: + + T = TypeVar('T') # Can be anything + S = TypeVar('S', bound=str) # Can be any subtype of str + A = TypeVar('A', str, bytes) # Must be exactly str or bytes + Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well - as for generic function definitions. See :class:`Generic` for more + as for generic function and type alias definitions. + See :class:`Generic` for more information on generic types. Generic functions work as follows:: def repeat[T](x: T, n: int) -> Sequence[T]: @@ -1428,8 +1429,9 @@ without the dedicated syntax, as documented below. through the :ref:`type parameter syntax ` and when ``infer_variance=True`` is passed. Manually created type variables may be explicitly marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` and :pep:`695` for more - details. By default, type variables are invariant. + ``covariant=True`` or ``contravariant=True``. + By default, manually created type variables are invariant. + See :pep:`484` and :pep:`695` for more details. Bound type variables and constrained type variables have different semantics in several important ways. Using a *bound* type variable means @@ -1449,13 +1451,13 @@ without the dedicated syntax, as documented below. Type variables can be bound to concrete types, abstract types (ABCs or protocols), and even unions of types:: - U = TypeVar('U', bound=str|bytes) # Can be any subtype of the union str|bytes - V = TypeVar('V', bound=SupportsAbs) # Can be anything with an __abs__ method - # Can be anything with an __abs__ method - def print_abs[V: SupportsAbs](arg: V) -> None: + def print_abs[T: SupportsAbs](arg: T) -> None: print("Absolute value:", abs(arg)) + U = TypeVar('U', bound=str|bytes) # Can be any subtype of the union str|bytes + V = TypeVar('V', bound=SupportsAbs) # Can be anything with an __abs__ method + .. _typing-constrained-typevar: Using a *constrained* type variable, however, means that the ``TypeVar`` @@ -1469,8 +1471,7 @@ without the dedicated syntax, as documented below. c = concatenate('one', b'two') # error: type variable 'A' can be either str or bytes in a function call, but not both - At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issubclass` should not be used with types. + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. .. attribute:: __name__ @@ -1478,11 +1479,11 @@ without the dedicated syntax, as documented below. .. attribute:: __covariant__ - Whether the type variable is covariant. + Whether the type var has been explicitly marked as covariant. .. attribute:: __contravariant__ - Whether the type variable is contravariant. + Whether the type var has been explicitly marked as contravariant. .. attribute:: __infer_variance__ diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 095ce3756305cc..9ee36b74df3788 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -443,42 +443,42 @@ static PyMethodDef typevar_methods[] = { PyDoc_STRVAR(typevar_doc, "Type variable.\n\ \n\ -Usage::\n\ +The preferred way to construct a type variable is via the dedicated syntax\n\ +for :ref:`generic functions `,\n\ +:ref:`generic classes `, and\n\ +:ref:`generic type aliases `::\n\ \n\ - T = TypeVar('T') # Can be anything\n\ - S = TypeVar('S', bound=str) # Can be any subtype of str\n\ - A = TypeVar('A', str, bytes) # Must be str or bytes\n\ + class Sequence[T]: # T is a TypeVar\n\ + ...\n\ \n\ -The syntax for generic functions, classes, and type aliases can\n\ -be used to create type variables::\n\ +This syntax can also be used to create bound and constrained type\n\ +variables::\n\ \n\ - class Sequence[T]: # T is a TypeVar\n\ - ...\n\ + class StrSequence[S: str]: # S is a TypeVar bound to str\n\ + ...\n\ \n\ -Type variables exist primarily for the benefit of static type\n\ -checkers. They serve as the parameters for generic types as well\n\ -as for generic function definitions. See class Generic for more\n\ -information on generic types. Generic functions work as follows:\n\ \n\ - def repeat[T](x: T, n: int) -> Sequence[T]:\n\ - '''Return a list containing n references to x.'''\n\ - return [x]*n\n\ + class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes\n\ + ...\n\ \n\ - def print_capitalized[S: str](x: S) -> S:\n\ - '''Print x capitalized, and return x.'''\n\ - print(x.capitalize())\n\ - return x\n\ +However, if desired, reusable type variables can also be constructed\n\ +manually, like so::\n\ \n\ - def concatenate[A: (str, bytes)](x: A, y: A) -> A:\n\ - '''Add two strings or bytes objects together.'''\n\ - return x + y\n\ + T = TypeVar('T') # Can be anything\n\ + S = TypeVar('S', bound=str) # Can be any subtype of str\n\ + A = TypeVar('A', str, bytes) # Must be exactly str or bytes\n\ +\n\ +Type variables exist primarily for the benefit of static type\n\ +checkers. They serve as the parameters for generic types as well\n\ +as for generic function and type alias definitions.\n\ \n\ The variance of type variables is inferred by type checkers when they are created\n\ through the type parameter syntax and when\n\ ``infer_variance=True`` is passed.\n\ -Manually created type variables may be explicitly marked covariant or contravariant by passing\n\ -``covariant=True`` or ``contravariant=True``. See PEP 484 and PEP 695 for more\n\ -details. By default, type variables are invariant.\n\ +Manually created type variables may be explicitly marked covariant or\n\ +contravariant by passing ``covariant=True`` or ``contravariant=True``.\n\ +By default, manually created type variables are invariant. See PEP 484\n\ +and PEP 695 for more details.\n\ "); static PyType_Slot typevar_slots[] = { From c94ee7ce88d9eb9b6937870c9b2b98c0eeab23a6 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 18:49:26 -0700 Subject: [PATCH 37/56] Fix alignment --- Doc/library/typing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 74a9c03a1d4b3f..e1cd734e4678d4 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1597,7 +1597,7 @@ without the dedicated syntax, as documented below. However, note that at most one type variable tuple may appear in a single list of type arguments or type parameters:: - x: tuple[*Ts, *Ts] # Not valid + x: tuple[*Ts, *Ts] # Not valid class Array[*Shape, *Shape]: # Not valid pass From 16303421c5fcaa10ce8d07234cf4101e22dd60e9 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 18:50:32 -0700 Subject: [PATCH 38/56] doctest maybe --- Doc/library/typing.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e1cd734e4678d4..ccffd666d45814 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1779,8 +1779,10 @@ without the dedicated syntax, as documented below. Example:: + .. doctest:: + >>> type ListOrSet[T] = list[T] | set[T] - >>> Alias.__type_params__ + >>> ListOrSet.__type_params__ (T,) >>> type NotGeneric = int >>> NotGeneric.__type_params__ @@ -1794,6 +1796,8 @@ without the dedicated syntax, as documented below. Example:: + .. doctest:: + >>> type Mutually = Recursive >>> type Recursive = Mutually >>> Mutually.__value__ From aade45a40c68dca73fb111fa602c50f42a35c30d Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 18:55:06 -0700 Subject: [PATCH 39/56] More Alex suggestions --- Doc/library/typing.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index ccffd666d45814..ef67d593de69b6 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1800,6 +1800,10 @@ without the dedicated syntax, as documented below. >>> type Mutually = Recursive >>> type Recursive = Mutually + >>> Mutually + Mutually + >>> Recursive + Recursive >>> Mutually.__value__ Recursive >>> Recursive.__value__ @@ -2134,7 +2138,7 @@ These are not used in annotations. They are building blocks for declaring types. group: list[T] To create a generic ``TypedDict`` that is compatible with Python 3.11 - or lower, use :class:`Generic` explicitly:: + or lower, inherit from :class:`Generic` explicitly:: class Group(TypedDict, Generic[T]): key: T From 57b9992df0bf067a5370209ffe798b2086bc50e2 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 19:02:44 -0700 Subject: [PATCH 40/56] Feedback from Carl --- Doc/library/typing.rst | 7 ++++--- Doc/reference/compound_stmts.rst | 20 ++++++++++++-------- Doc/reference/executionmodel.rst | 12 ++++++------ 3 files changed, 22 insertions(+), 17 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index ef67d593de69b6..bb663f6e5967ee 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1361,7 +1361,8 @@ without the dedicated syntax, as documented below. Here the brackets after the function name indicate a :ref:`generic function `. - Generic classes can also be declared by explicitly inheriting from + For backwards compatibility reasons, generic classes can also be + declared by explicitly inheriting from ``Generic``. In this case, the type parameters must be declared separately:: @@ -1497,7 +1498,7 @@ without the dedicated syntax, as documented below. .. versionchanged:: 3.12 - For type variables created through the new type parameter syntax, + For type variables created through :ref:`type parameter syntax `, the bound is evaluated only when the attribute is accessed, not when the type variable is created. @@ -1507,7 +1508,7 @@ without the dedicated syntax, as documented below. .. versionchanged:: 3.12 - For type variables created through the new type parameter syntax, + For type variables created through :ref:`type parameter syntax `, the constraints are evaluated only when the attribute is accessed, not when the type variable is created. diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index b36fcc605eec6f..9bbd221dfe0676 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1262,7 +1262,7 @@ This indicates to static type checkers that the function is generic. At runtime, the type parameters can be retrieved from the function's ``__type_params__`` attribute. See :ref:`generic-functions` for more. -.. versionadded:: 3.12 +.. versionchanged:: 3.12 Type parameters are new in version 3.12. .. index:: @@ -1449,7 +1449,7 @@ This indicates to static type checkers that the class is generic. At runtime, the type parameters can be retrieved from the class's ``__type_params__`` attribute. See :ref:`generic-classes` for more. -.. versionadded:: 3.12 +.. versionchanged:: 3.12 Type parameters are new in version 3.12. **Programmer's note:** Variables defined in the class definition are class @@ -1654,8 +1654,7 @@ are accessible within the scope of the generic object, but not elsewhere. Thus, after a declaration ``def func[T](): ...``, the name ``T`` is not in the module scope. Below, the semantics of generic objects are described with more precision. The scope of type parameters is modeled with a special -function (technically, an :ref:`annotation scope `, -represented with the pseudo-keyword ``def'``) that +function (technically, an :ref:`annotation scope `) that wraps the creation of the generic object. Generic functions, classes, and type aliases have a :attr:`__type_params__` @@ -1723,8 +1722,9 @@ This syntax is equivalent to:: return func func = TYPE_PARAMS_OF_func() -Here ``def'`` indicates an :ref:`annotation scope `. (One -other liberty is taken in the translation: the syntax does not go through +Here ``def'`` indicates an :ref:`annotation scope `. (Two +other liberties are taken in the translation: no function is actually bound +to the name ``TYPE_PARAMS_OF_func``, and the syntax does not go through attribute access on the :mod:`typing` module, but creates an instance of :data:`typing.TypeVar` directly.) @@ -1739,7 +1739,7 @@ as well as for additional flavors of type parameters:: def func[T: int, *Ts, **P](*args: *Ts, arg: Callable[P, T] = some_default): ... -Except for the lazy evaluation of the ``TypeVar`` bound, this is equivalent to:: +Except for the lazy evaluation of the :class:`~typing.TypeVar` bound, this is equivalent to:: DEFAULT_OF_arg = some_default @@ -1760,6 +1760,9 @@ Except for the lazy evaluation of the ``TypeVar`` bound, this is equivalent to:: return func func = decorator(TYPE_PARAMS_OF_func()) +(The uppercased names like ``DEFAULT_OF_arg`` are not actually +bound at runtime.) + .. _generic-classes: Generic classes @@ -1779,7 +1782,8 @@ This syntax is equivalent to:: return Bag Bag = TYPE_PARAMS_OF_Bag() -Here again ``def'`` indicates an :ref:`annotation scope `. +Here again ``def'`` (not a real keyword) indicates an +:ref:`annotation scope `. Generic classes implicitly inherit from :data:`typing.Generic`. The base classes and keyword arguments of generic classes are diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 376918403f7939..f622e4bccb63a3 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -167,8 +167,8 @@ that unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods. This includes -comprehensions and generator expressions since they are implemented using a -function scope, but it does not include :ref:`annotation scopes `, +comprehensions and generator expressions, but it does not include +:ref:`annotation scopes `, which have access to their enclosing class scopes. This means that the following will fail:: @@ -199,9 +199,9 @@ Annotation scopes are used in the following contexts: * Type parameter lists for generic type aliases. * Type parameter lists for generic functions. The function's annotations are - executed within the type scope, but its defaults and decorators are not. + executed within the annotation scope, but its defaults and decorators are not. * Type parameter lists for generic classes. The class's base classes and - keyword arguments are executed within the type scope, but its decorators are not. + keyword arguments are executed within the annotation scope, but its decorators are not. * The bounds and constraints for type variables. * The value of type aliases. @@ -214,10 +214,10 @@ Annotation scopes differ from function scopes in the following ways: functions defined within classes cannot access names defined in the class scope.) * Expressions in annotation scopes cannot contain :keyword:`yield`, ``yield from``, :keyword:`await`, or :token:`:= <~python-grammar:expression>` expressions. (These - expressions are allowed in other scopes contained within the type scope.) + expressions are allowed in other scopes contained within the annotation scope.) * Names defined in annotation scopes cannot be rebound with :keyword:`nonlocal` statements in inner scopes. This includes only type parameters, as no other - syntactic elements that can appear within type scopes can introduce new names. + syntactic elements that can appear within annotation scopes can introduce new names. * While annotation scopes internally have a name, that name is not reflected in the ``__qualname__`` of objects defined within the scope. Instead, the ``__qualname__`` of such objects is as if the object was defined in the enclosing scope. From 7953d2cfc8035a234fc25e8d8cee255e0ee55897 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 20:19:13 -0700 Subject: [PATCH 41/56] Section on lazy eval --- Doc/reference/executionmodel.rst | 50 ++++++++++++++++++++++++++++---- 1 file changed, 45 insertions(+), 5 deletions(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index f622e4bccb63a3..4380da532db7ba 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -197,13 +197,14 @@ annotation scopes in Python 3.13 when :pep:`649` is implemented. Annotation scopes are used in the following contexts: -* Type parameter lists for generic type aliases. -* Type parameter lists for generic functions. The function's annotations are +* Type parameter lists for :ref:`generic type aliases `. +* Type parameter lists for :ref:`generic functions `. The function's annotations are executed within the annotation scope, but its defaults and decorators are not. -* Type parameter lists for generic classes. The class's base classes and +* Type parameter lists for :ref:`generic classes `. The class's base classes and keyword arguments are executed within the annotation scope, but its decorators are not. -* The bounds and constraints for type variables. -* The value of type aliases. +* The bounds and constraints for type variables + (:ref:`lazily evaluated `). +* The value of type aliases (:ref:`lazily evaluated `). Annotation scopes differ from function scopes in the following ways: @@ -225,6 +226,45 @@ Annotation scopes differ from function scopes in the following ways: .. versionadded:: 3.12 Annotation scopes were introduced in Python 3.12 as part of :pep:`695`. +.. _lazy_evaluation: + +Lazy evaluation +--------------- + +The values of type aliases created through the :keyword:`type` statement and +the bounds and constraints of type variables created through the +:ref:`type parameter syntax ` are *lazily evaluated*. This means +that they are not evaluated when the type alias or type variable is created, +but only when evaluation is necessary to resolve an attribute access. + +Example: + +.. doctest:: + + >>> type Alias = 1/0 + >>> Alias.__value__ + Traceback (most recent call last): + ... + ZeroDivisionError: division by zero + +Here the exception is raised only when the ``__value__`` attribute is accessed. + +This behavior is primarily useful for references to types that have not +yet been defined when the type alias or type variable is created. For example, +lazy evaluation enables creation of mutually recursive type aliases:: + + from typing import Literal + + type SimpleExpr = int | Parenthesized + type Parenthesized = tuple[Literal["("], Expr, Literal[")"]] + type Expr = SimpleExpr | tuple[SimpleExpr, Literal["+", "-"], Expr] + +Lazily evaluated values are evaluated in :ref:`annotation scope `, +which means that names that appear inside the lazily evaluated value are looked up +as if they were used in the immediately enclosing scope. + +.. versionadded:: 3.12 + .. _restrict_exec: Builtins and restricted execution From 9128c727e1298f352788a2fbc0b8787eb0e6ec79 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 20:24:36 -0700 Subject: [PATCH 42/56] More cross-refs --- Doc/library/typing.rst | 4 ++-- Doc/reference/compound_stmts.rst | 17 ++++++++++++----- Doc/reference/executionmodel.rst | 2 +- Doc/reference/simple_stmts.rst | 4 ++-- 4 files changed, 17 insertions(+), 10 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index bb663f6e5967ee..87a096c7f6e8c6 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1500,7 +1500,7 @@ without the dedicated syntax, as documented below. For type variables created through :ref:`type parameter syntax `, the bound is evaluated only when the attribute is accessed, not when - the type variable is created. + the type variable is created (see :ref:`lazy-evaluation`). .. attribute:: __constraints__ @@ -1510,7 +1510,7 @@ without the dedicated syntax, as documented below. For type variables created through :ref:`type parameter syntax `, the constraints are evaluated only when the attribute is accessed, not when - the type variable is created. + the type variable is created (see :ref:`lazy-evaluation`). .. versionchanged:: 3.12 diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 9bbd221dfe0676..06fcf4cac4f983 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1739,7 +1739,8 @@ as well as for additional flavors of type parameters:: def func[T: int, *Ts, **P](*args: *Ts, arg: Callable[P, T] = some_default): ... -Except for the lazy evaluation of the :class:`~typing.TypeVar` bound, this is equivalent to:: +Except for the :ref:`lazy evaluation ` of the +:class:`~typing.TypeVar` bound, this is equivalent to:: DEFAULT_OF_arg = some_default @@ -1760,8 +1761,8 @@ Except for the lazy evaluation of the :class:`~typing.TypeVar` bound, this is eq return func func = decorator(TYPE_PARAMS_OF_func()) -(The uppercased names like ``DEFAULT_OF_arg`` are not actually -bound at runtime.) +The capitalized names like ``DEFAULT_OF_arg`` are not actually +bound at runtime. .. _generic-classes: @@ -1783,7 +1784,8 @@ This syntax is equivalent to:: Bag = TYPE_PARAMS_OF_Bag() Here again ``def'`` (not a real keyword) indicates an -:ref:`annotation scope `. +:ref:`annotation scope `, and the name +``TYPE_PARAMS_OF_Bag`` is not actually bound at runtime. Generic classes implicitly inherit from :data:`typing.Generic`. The base classes and keyword arguments of generic classes are @@ -1813,7 +1815,8 @@ The :keyword:`type` statement can also be used to create a generic type alias:: type ListOrSet[T] = list[T] | set[T] -Except for the lazy evaluation of the value, this is equivalent to:: +Except for the :ref:`lazy evaluation ` of the value, +this is equivalent to:: def' TYPE_PARAMS_OF_ListOrSet(): T = typing.TypeVar("T") @@ -1824,6 +1827,10 @@ Except for the lazy evaluation of the value, this is equivalent to:: return typing.TypeAliasType("ListOrSet", VALUE_OF_ListOrSet(), type_params=(T,)) ListOrSet = TYPE_PARAMS_OF_ListOrSet() +Here, ``def'`` (not a real keyword) indicates an +:ref:`annotation scope `. The capitalized names +like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime. + .. rubric:: Footnotes .. [#] The exception is propagated to the invocation stack unless diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 4380da532db7ba..493959a1fc1531 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -226,7 +226,7 @@ Annotation scopes differ from function scopes in the following ways: .. versionadded:: 3.12 Annotation scopes were introduced in Python 3.12 as part of :pep:`695`. -.. _lazy_evaluation: +.. _lazy-evaluation: Lazy evaluation --------------- diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 902d969ce4e272..920b65bfa0a7cc 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -1041,8 +1041,8 @@ This code is roughly equivalent to:: mostly like a function, but with several small differences. The value of the type alias is evaluated in the annotation scope. It is not evaluated when the type alias is created, but only when the value is accessed through the type alias's -``__value__`` attribute. This allows the type alias to refer to names that are -not yet defined. +``__value__`` attribute (see :ref:`lazy-evaluation`). +This allows the type alias to refer to names that are not yet defined. Type aliases may be made generic by adding a :ref:`type parameter list ` after the name. See :ref:`generic-type-aliases` for more. From 0511b6aaac7f0bd2ae6115ea23d1cd341a415a58 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 22 May 2023 20:40:09 -0700 Subject: [PATCH 43/56] Can't have nice things --- Doc/library/typing.rst | 62 +++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 87a096c7f6e8c6..43f9428daaf835 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -998,7 +998,7 @@ These can be used as types in annotations using ``[]``, each having a unique syn * :class:`ParamSpec` and :class:`Callable`. -.. class:: Type[CT_co] +.. class:: Type(Generic[CT_co]) A variable annotated with ``C`` may accept a value of type ``C``. In contrast, a variable annotated with ``Type[C]`` may accept values that are @@ -2215,7 +2215,7 @@ Generic concrete collections Corresponding to built-in types """"""""""""""""""""""""""""""" -.. class:: Dict[KT, VT](dict, MutableMapping[KT, VT]) +.. class:: Dict(dict, MutableMapping[KT, VT]) A generic version of :class:`dict`. Useful for annotating return types. To annotate arguments it is preferred @@ -2230,7 +2230,7 @@ Corresponding to built-in types :class:`builtins.dict ` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: List[T](list, MutableSequence[T]) +.. class:: List(list, MutableSequence[T]) Generic version of :class:`list`. Useful for annotating return types. To annotate arguments it is preferred @@ -2249,7 +2249,7 @@ Corresponding to built-in types :class:`builtins.list ` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Set[T](set, MutableSet[T]) +.. class:: Set(set, MutableSet[T]) A generic version of :class:`builtins.set `. Useful for annotating return types. To annotate arguments it is preferred @@ -2259,7 +2259,7 @@ Corresponding to built-in types :class:`builtins.set ` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: FrozenSet[T_co](frozenset, AbstractSet[T_co]) +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) A generic version of :class:`builtins.frozenset `. @@ -2273,7 +2273,7 @@ Corresponding to built-in types Corresponding to types in :mod:`collections` """""""""""""""""""""""""""""""""""""""""""" -.. class:: DefaultDict[KT, VT](collections.defaultdict, MutableMapping[KT, VT]) +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) A generic version of :class:`collections.defaultdict`. @@ -2283,7 +2283,7 @@ Corresponding to types in :mod:`collections` :class:`collections.defaultdict` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: OrderedDict[KT, VT](collections.OrderedDict, MutableMapping[KT, VT]) +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) A generic version of :class:`collections.OrderedDict`. @@ -2293,7 +2293,7 @@ Corresponding to types in :mod:`collections` :class:`collections.OrderedDict` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ChainMap[KT, VT](collections.ChainMap, MutableMapping[KT, VT]) +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) A generic version of :class:`collections.ChainMap`. @@ -2304,7 +2304,7 @@ Corresponding to types in :mod:`collections` :class:`collections.ChainMap` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Counter[T](collections.Counter, Dict[T, int]) +.. class:: Counter(collections.Counter, Dict[T, int]) A generic version of :class:`collections.Counter`. @@ -2315,7 +2315,7 @@ Corresponding to types in :mod:`collections` :class:`collections.Counter` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Deque[T](deque, MutableSequence[T]) +.. class:: Deque(deque, MutableSequence[T]) A generic version of :class:`collections.deque`. @@ -2386,7 +2386,7 @@ Abstract Base Classes Corresponding to collections in :mod:`collections.abc` """""""""""""""""""""""""""""""""""""""""""""""""""""" -.. class:: AbstractSet[T_co](Collection[T_co]) +.. class:: AbstractSet(Collection[T_co]) A generic version of :class:`collections.abc.Set`. @@ -2402,7 +2402,7 @@ Corresponding to collections in :mod:`collections.abc` .. deprecated-removed:: 3.9 3.14 Prefer :class:`collections.abc.Buffer`, or a union like ``bytes | bytearray | memoryview``. -.. class:: Collection[T_co](Sized, Iterable[T_co], Container[T_co]) +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) A generic version of :class:`collections.abc.Collection` @@ -2412,7 +2412,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Collection` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Container[T_co] +.. class:: Container(Generic[T_co]) A generic version of :class:`collections.abc.Container`. @@ -2420,7 +2420,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Container` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ItemsView[KT_co, VT_co](MappingView, AbstractSet[tuple[KT_co, VT_co]]) +.. class:: ItemsView(MappingView, AbstractSet[tuple[KT_co, VT_co]]) A generic version of :class:`collections.abc.ItemsView`. @@ -2428,7 +2428,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.ItemsView` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: KeysView[KT_co](MappingView, AbstractSet[KT_co]) +.. class:: KeysView(MappingView, AbstractSet[KT_co]) A generic version of :class:`collections.abc.KeysView`. @@ -2436,7 +2436,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.KeysView` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Mapping[KT, VT_co](Collection[KT]) +.. class:: Mapping(Collection[KT], Generic[KT, VT_co]) A generic version of :class:`collections.abc.Mapping`. This type can be used as follows:: @@ -2456,7 +2456,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.MappingView` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: MutableMapping[KT, VT](Mapping[KT, VT]) +.. class:: MutableMapping(Mapping[KT, VT]) A generic version of :class:`collections.abc.MutableMapping`. @@ -2465,7 +2465,7 @@ Corresponding to collections in :mod:`collections.abc` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: MutableSequence[T](Sequence[T]) +.. class:: MutableSequence(Sequence[T]) A generic version of :class:`collections.abc.MutableSequence`. @@ -2474,7 +2474,7 @@ Corresponding to collections in :mod:`collections.abc` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: MutableSet[T](AbstractSet[T]) +.. class:: MutableSet(AbstractSet[T]) A generic version of :class:`collections.abc.MutableSet`. @@ -2482,7 +2482,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.MutableSet` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Sequence[T_co](Reversible[T_co], Collection[T_co]) +.. class:: Sequence(Reversible[T_co], Collection[T_co]) A generic version of :class:`collections.abc.Sequence`. @@ -2490,7 +2490,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Sequence` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ValuesView[VT_co](MappingView, Collection[VT_co]) +.. class:: ValuesView(MappingView, Collection[VT_co]) A generic version of :class:`collections.abc.ValuesView`. @@ -2501,7 +2501,7 @@ Corresponding to collections in :mod:`collections.abc` Corresponding to other types in :mod:`collections.abc` """""""""""""""""""""""""""""""""""""""""""""""""""""" -.. class:: Iterable[T_co] +.. class:: Iterable(Generic[T_co]) A generic version of :class:`collections.abc.Iterable`. @@ -2509,7 +2509,7 @@ Corresponding to other types in :mod:`collections.abc` :class:`collections.abc.Iterable` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Iterator[T_co](Iterable[T_co]) +.. class:: Iterator(Iterable[T_co]) A generic version of :class:`collections.abc.Iterator`. @@ -2517,7 +2517,7 @@ Corresponding to other types in :mod:`collections.abc` :class:`collections.abc.Iterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Generator[YieldType, SendType, ReturnType](Iterator[YieldType]) +.. class:: Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType]) A generator can be annotated by the generic type ``Generator[YieldType, SendType, ReturnType]``. For example:: @@ -2559,7 +2559,7 @@ Corresponding to other types in :mod:`collections.abc` .. deprecated:: 3.12 Use :class:`collections.abc.Hashable` directly instead. -.. class:: Reversible[T_co](Iterable[T_co]) +.. class:: Reversible(Iterable[T_co]) A generic version of :class:`collections.abc.Reversible`. @@ -2577,7 +2577,7 @@ Corresponding to other types in :mod:`collections.abc` Asynchronous programming """""""""""""""""""""""" -.. class:: Coroutine[YieldType, SendType, ReturnType](Awaitable[ReturnType]) +.. class:: Coroutine(Awaitable[ReturnType], Generic[YieldType, SendType, ReturnType]) A generic version of :class:`collections.abc.Coroutine`. The variance and order of type variables @@ -2595,7 +2595,7 @@ Asynchronous programming :class:`collections.abc.Coroutine` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncGenerator[YieldType, SendType](AsyncIterator[YieldType]) +.. class:: AsyncGenerator(AsyncIterator[YieldType], Generic[YieldType, SendType]) An async generator can be annotated by the generic type ``AsyncGenerator[YieldType, SendType]``. For example:: @@ -2633,7 +2633,7 @@ Asynchronous programming now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncIterable[T_co] +.. class:: AsyncIterable(Generic[T_co]) A generic version of :class:`collections.abc.AsyncIterable`. @@ -2643,7 +2643,7 @@ Asynchronous programming :class:`collections.abc.AsyncIterable` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncIterator[T_co](AsyncIterable[T_co]) +.. class:: AsyncIterator(AsyncIterable[T_co]) A generic version of :class:`collections.abc.AsyncIterator`. @@ -2667,7 +2667,7 @@ Asynchronous programming Context manager types """"""""""""""""""""" -.. class:: ContextManager[T_co] +.. class:: ContextManager(Generic[T_co]) A generic version of :class:`contextlib.AbstractContextManager`. @@ -2679,7 +2679,7 @@ Context manager types now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncContextManager[T_co] +.. class:: AsyncContextManager(Generic[T_co]) A generic version of :class:`contextlib.AbstractAsyncContextManager`. From 4fd3f02182e4bf89c21ac65dd6c7789cd7936246 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 24 May 2023 08:44:33 -0700 Subject: [PATCH 44/56] Respond to some feedback --- Doc/library/typing.rst | 2 +- Doc/reference/compound_stmts.rst | 31 +++++++++++++++---------------- Doc/reference/simple_stmts.rst | 4 ++-- 3 files changed, 18 insertions(+), 19 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 43f9428daaf835..f41ed76094d607 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1361,7 +1361,7 @@ without the dedicated syntax, as documented below. Here the brackets after the function name indicate a :ref:`generic function `. - For backwards compatibility reasons, generic classes can also be + For backwards compatibility, generic classes can also be declared by explicitly inheriting from ``Generic``. In this case, the type parameters must be declared separately:: diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 06fcf4cac4f983..eabb941939b7dd 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1651,7 +1651,7 @@ non-generic counterparts. Type parameters are declared in square brackets (``[]``) immediately after the name of the function, class, or type alias. The type parameters are accessible within the scope of the generic object, but not elsewhere. -Thus, after a declaration ``def func[T](): ...``, the name ``T`` is not in +Thus, after a declaration ``def func[T](): pass``, the name ``T`` is not in the module scope. Below, the semantics of generic objects are described with more precision. The scope of type parameters is modeled with a special function (technically, an :ref:`annotation scope `) that @@ -1663,7 +1663,7 @@ attribute listing their type parameters. Type parameters come in three kinds: * :data:`typing.TypeVar`, introduced by a plain name (e.g., ``T``). Semantically, this - stands for a single type. + represents a single type to a type checker. * :data:`typing.TypeVarTuple`, introduced by a name prefixed with a single asterisk (e.g., ``*Ts``). Semantically, this stands for a tuple of any number of types. @@ -1672,8 +1672,7 @@ Type parameters come in three kinds: :data:`typing.TypeVar` declarations can define *bounds* and *constraints* with a colon (``:``) followed by an expression. A single expression after the colon -indicates a bound (e.g. ``T: int``). The expression should be a type (though this -is not enforced at runtime, only by static type checkers) and semantically, this means +indicates a bound (e.g. ``T: int``). Semantically, this means that the :data:`!typing.TypeVar` can only represent types that are a subtype of this bound. A parenthesized tuple of expressions after the colon indicates a set of constraints (e.g. ``T: (str, bytes)``). Each member of the tuple should be a @@ -1715,16 +1714,16 @@ Generic functions are declared as follows:: This syntax is equivalent to:: - def' TYPE_PARAMS_OF_func(): + annotation-def TYPE_PARAMS_OF_func(): T = typing.TypeVar("T") def func(arg: T): ... func.__type_params__ = (T,) return func func = TYPE_PARAMS_OF_func() -Here ``def'`` indicates an :ref:`annotation scope `. (Two -other liberties are taken in the translation: no function is actually bound -to the name ``TYPE_PARAMS_OF_func``, and the syntax does not go through +Here ``annotation-def`` indicates an :ref:`annotation scope `, +which is not actually bound to any name at runtime. (One +other liberties are taken in the translation: the syntax does not go through attribute access on the :mod:`typing` module, but creates an instance of :data:`typing.TypeVar` directly.) @@ -1744,9 +1743,9 @@ Except for the :ref:`lazy evaluation ` of the DEFAULT_OF_arg = some_default - def' TYPE_PARAMS_OF_func(): + annotation-def TYPE_PARAMS_OF_func(): - def' BOUND_OF_T(): + annotation-def BOUND_OF_T(): return int # In reality, BOUND_OF_T() is evaluated only on demand. T = typing.TypeVar("T", bound=BOUND_OF_T()) @@ -1775,7 +1774,7 @@ Generic classes are declared as follows:: This syntax is equivalent to:: - def' TYPE_PARAMS_OF_Bag(): + annotation-def TYPE_PARAMS_OF_Bag(): T = typing.TypeVar("T") class Bag(typing.Generic[T]): __type_params__ = (T,) @@ -1783,7 +1782,7 @@ This syntax is equivalent to:: return Bag Bag = TYPE_PARAMS_OF_Bag() -Here again ``def'`` (not a real keyword) indicates an +Here again ``annotation-def`` (not a real keyword) indicates an :ref:`annotation scope `, and the name ``TYPE_PARAMS_OF_Bag`` is not actually bound at runtime. @@ -1798,7 +1797,7 @@ by this example:: This is equivalent to:: - def' TYPE_PARAMS_OF_Bag(): + annotation-def TYPE_PARAMS_OF_Bag(): T = typing.TypeVar("T") class Bag(Base[T], typing.Generic[T], arg=T): __type_params__ = (T,) @@ -1818,16 +1817,16 @@ The :keyword:`type` statement can also be used to create a generic type alias:: Except for the :ref:`lazy evaluation ` of the value, this is equivalent to:: - def' TYPE_PARAMS_OF_ListOrSet(): + annotation-def TYPE_PARAMS_OF_ListOrSet(): T = typing.TypeVar("T") - def' VALUE_OF_ListOrSet(): + annotation-def VALUE_OF_ListOrSet(): return list[T] | set[T] # In reality, the value is lazily evaluated return typing.TypeAliasType("ListOrSet", VALUE_OF_ListOrSet(), type_params=(T,)) ListOrSet = TYPE_PARAMS_OF_ListOrSet() -Here, ``def'`` (not a real keyword) indicates an +Here, ``annotation-def`` (not a real keyword) indicates an :ref:`annotation scope `. The capitalized names like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime. diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 920b65bfa0a7cc..a250501483f18c 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -1033,11 +1033,11 @@ For example, the following statement creates a type alias:: This code is roughly equivalent to:: - def' VALUE_OF_Point(): + annotation-def VALUE_OF_Point(): return tuple[float, float] Point = typing.TypeAliasType("Point", VALUE_OF_Point()) -``def'`` indicates an :ref:`annotation scope `, which behaves +``annotation-def`` indicates an :ref:`annotation scope `, which behaves mostly like a function, but with several small differences. The value of the type alias is evaluated in the annotation scope. It is not evaluated when the type alias is created, but only when the value is accessed through the type alias's From fff0df4ece07d64a3e5ce66a146d4a21523db208 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 24 May 2023 08:52:31 -0700 Subject: [PATCH 45/56] Undo unnecessary typing.rst changes --- Doc/library/typing.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index f41ed76094d607..933263c2749a21 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2006,6 +2006,7 @@ These are not used in annotations. They are building blocks for declaring types. protocol. See :ref:`"What's new in Python 3.12" ` for more details. + .. class:: TypedDict(dict) Special construct to add type hints to a dictionary. @@ -2490,7 +2491,7 @@ Corresponding to collections in :mod:`collections.abc` :class:`collections.abc.Sequence` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: ValuesView(MappingView, Collection[VT_co]) +.. class:: ValuesView(MappingView, Collection[_VT_co]) A generic version of :class:`collections.abc.ValuesView`. @@ -2517,7 +2518,7 @@ Corresponding to other types in :mod:`collections.abc` :class:`collections.abc.Iterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType]) +.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type ``Generator[YieldType, SendType, ReturnType]``. For example:: @@ -2577,7 +2578,7 @@ Corresponding to other types in :mod:`collections.abc` Asynchronous programming """""""""""""""""""""""" -.. class:: Coroutine(Awaitable[ReturnType], Generic[YieldType, SendType, ReturnType]) +.. class:: Coroutine(Awaitable[V_co], Generic[T_co, T_contra, V_co]) A generic version of :class:`collections.abc.Coroutine`. The variance and order of type variables @@ -2595,7 +2596,7 @@ Asynchronous programming :class:`collections.abc.Coroutine` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: AsyncGenerator(AsyncIterator[YieldType], Generic[YieldType, SendType]) +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) An async generator can be annotated by the generic type ``AsyncGenerator[YieldType, SendType]``. For example:: @@ -2653,7 +2654,7 @@ Asynchronous programming :class:`collections.abc.AsyncIterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. -.. class:: Awaitable[T_co] +.. class:: Awaitable(Generic[T_co]) A generic version of :class:`collections.abc.Awaitable`. From 1c6807d3a334952c04422682de148725b06acdba Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 24 May 2023 08:54:06 -0700 Subject: [PATCH 46/56] more words --- Doc/reference/compound_stmts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index eabb941939b7dd..53b7f6d0577a95 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1651,7 +1651,7 @@ non-generic counterparts. Type parameters are declared in square brackets (``[]``) immediately after the name of the function, class, or type alias. The type parameters are accessible within the scope of the generic object, but not elsewhere. -Thus, after a declaration ``def func[T](): pass``, the name ``T`` is not in +Thus, after a declaration ``def func[T](): pass``, the name ``T`` is not available in the module scope. Below, the semantics of generic objects are described with more precision. The scope of type parameters is modeled with a special function (technically, an :ref:`annotation scope `) that From 638647563f8261d6d60d0e4a77fda672167bef82 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 24 May 2023 08:56:20 -0700 Subject: [PATCH 47/56] or --- Doc/library/typing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d2399f790e970d..55effe7e7d7851 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1427,7 +1427,7 @@ without the dedicated syntax, as documented below. cannot be both bound *and* constrained. The variance of type variables is inferred by type checkers when they are created - through the :ref:`type parameter syntax ` and when + through the :ref:`type parameter syntax ` or when ``infer_variance=True`` is passed. Manually created type variables may be explicitly marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. From cd4206c7589ee529b0099cb02c604a7190c8e6a8 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 24 May 2023 12:21:48 -0700 Subject: [PATCH 48/56] Update Doc/reference/compound_stmts.rst Co-authored-by: Alex Waygood --- Doc/reference/compound_stmts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 53b7f6d0577a95..412d3692e38a64 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1723,7 +1723,7 @@ This syntax is equivalent to:: Here ``annotation-def`` indicates an :ref:`annotation scope `, which is not actually bound to any name at runtime. (One -other liberties are taken in the translation: the syntax does not go through +other liberty is taken in the translation: the syntax does not go through attribute access on the :mod:`typing` module, but creates an instance of :data:`typing.TypeVar` directly.) From d7e1c6c79e151c8b0769c5661e9fb740ba9664db Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 25 May 2023 17:37:29 -0700 Subject: [PATCH 49/56] Apply suggestions from code review Co-authored-by: Alex Waygood --- Doc/reference/executionmodel.rst | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 493959a1fc1531..ed157e4cff59f5 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -198,9 +198,11 @@ annotation scopes in Python 3.13 when :pep:`649` is implemented. Annotation scopes are used in the following contexts: * Type parameter lists for :ref:`generic type aliases `. -* Type parameter lists for :ref:`generic functions `. The function's annotations are +* Type parameter lists for :ref:`generic functions `. + A generic function's annotations are executed within the annotation scope, but its defaults and decorators are not. -* Type parameter lists for :ref:`generic classes `. The class's base classes and +* Type parameter lists for :ref:`generic classes `. + A generic class's base classes and keyword arguments are executed within the annotation scope, but its decorators are not. * The bounds and constraints for type variables (:ref:`lazily evaluated `). @@ -219,9 +221,10 @@ Annotation scopes differ from function scopes in the following ways: * Names defined in annotation scopes cannot be rebound with :keyword:`nonlocal` statements in inner scopes. This includes only type parameters, as no other syntactic elements that can appear within annotation scopes can introduce new names. -* While annotation scopes internally have a name, that name is not reflected in the - ``__qualname__`` of objects defined within the scope. Instead, the ``__qualname__`` - of such objects is as if the object was defined in the enclosing scope. +* While annotation scopes have an internal name, that name is not reflected in the + :term:`__qualname__ ` of objects defined within the scope. + Instead, the ``__qualname__`` + of such objects is as if the object were defined in the enclosing scope. .. versionadded:: 3.12 Annotation scopes were introduced in Python 3.12 as part of :pep:`695`. @@ -231,11 +234,12 @@ Annotation scopes differ from function scopes in the following ways: Lazy evaluation --------------- -The values of type aliases created through the :keyword:`type` statement and -the bounds and constraints of type variables created through the -:ref:`type parameter syntax ` are *lazily evaluated*. This means -that they are not evaluated when the type alias or type variable is created, -but only when evaluation is necessary to resolve an attribute access. +The values of type aliases created through the :keyword:`type` statement are +*lazily evaluated*. The same applies to the bounds and constraints of type +variables created through the :ref:`type parameter syntax `. +This means that they are not evaluated when the type alias or type variable is +created. Instead, they are only evaluated when doing so is necessary to resolve +an attribute access. Example: From f8789707f56fb77c424e78161f885e6691fd9c16 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 25 May 2023 17:45:03 -0700 Subject: [PATCH 50/56] More feedback from Alex --- Doc/reference/executionmodel.rst | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index ed157e4cff59f5..aa1754e267bb5e 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -211,13 +211,14 @@ Annotation scopes are used in the following contexts: Annotation scopes differ from function scopes in the following ways: * If an annotation scope is immediately within a class scope, or within another - annotation scope that is immediately within a class scope, names defined in the - class scope can be accessed from within the annotation scope, as if the code in the - annotation scope was executing directly within the class scope. (By contrast, regular - functions defined within classes cannot access names defined in the class scope.) + class scope can be accessed from within the annotation scope. The code in the + annotation scope can use names defined in the class scope as if it were + executed directly within the class body. This contrasts with regular + functions defined within classes, which cannot access names defined in the class scope. * Expressions in annotation scopes cannot contain :keyword:`yield`, ``yield from``, - :keyword:`await`, or :token:`:= <~python-grammar:expression>` expressions. (These - expressions are allowed in other scopes contained within the annotation scope.) + :keyword:`await`, or :token:`:= ` + expressions. (These expressions are allowed in other scopes contained within the + annotation scope.) * Names defined in annotation scopes cannot be rebound with :keyword:`nonlocal` statements in inner scopes. This includes only type parameters, as no other syntactic elements that can appear within annotation scopes can introduce new names. @@ -250,8 +251,16 @@ Example: Traceback (most recent call last): ... ZeroDivisionError: division by zero + >>> def func[T: 1/0](): pass + >>> T, = func.__type_params__ + >>> T.__bound__ + Traceback (most recent call last): + ... + ZeroDivisionError: division by zero -Here the exception is raised only when the ``__value__`` attribute is accessed. +Here the exception is raised only when the ``__value__`` attribute +of the type alias or the ``__bound__`` attribute of the type variable +is accessed. This behavior is primarily useful for references to types that have not yet been defined when the type alias or type variable is created. For example, From a4bd32a8f7e7bea4fd397a2b728330c68d869b91 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 26 May 2023 07:16:35 -0700 Subject: [PATCH 51/56] fixes --- Doc/reference/executionmodel.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index aa1754e267bb5e..c7a48a5793a447 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -211,7 +211,7 @@ Annotation scopes are used in the following contexts: Annotation scopes differ from function scopes in the following ways: * If an annotation scope is immediately within a class scope, or within another - class scope can be accessed from within the annotation scope. The code in the + class scope, the code in the annotation scope can use names defined in the class scope as if it were executed directly within the class body. This contrasts with regular functions defined within classes, which cannot access names defined in the class scope. @@ -252,7 +252,7 @@ Example: ... ZeroDivisionError: division by zero >>> def func[T: 1/0](): pass - >>> T, = func.__type_params__ + >>> T = func.__type_params__[0] >>> T.__bound__ Traceback (most recent call last): ... From 7a709d74f9219a656a97f732e7ac819d98cbaffc Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 26 May 2023 08:45:27 -0700 Subject: [PATCH 52/56] Rewrite and extend some docstrings --- Doc/reference/executionmodel.rst | 5 ++- Doc/whatsnew/3.12.rst | 12 +++--- Objects/typevarobject.c | 69 ++++++++++++++++++++------------ 3 files changed, 54 insertions(+), 32 deletions(-) diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index c7a48a5793a447..038be37884b43f 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -210,8 +210,9 @@ Annotation scopes are used in the following contexts: Annotation scopes differ from function scopes in the following ways: -* If an annotation scope is immediately within a class scope, or within another - class scope, the code in the +* Annotation scopes have access to their enclosing class namespace. + If an annotation scope is immediately within a class scope, or within another + annotation scope that is immediately within a class scope, the code in the annotation scope can use names defined in the class scope as if it were executed directly within the class body. This contrasts with regular functions defined within classes, which cannot access names defined in the class scope. diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 2ce15f2369fa18..18973d91f749fc 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -296,7 +296,7 @@ variance. def append(self, element: T) -> None: ... -In addition, the PEP introduces a new way to write :ref:`type aliases ` +In addition, the PEP introduces a new way to declare :ref:`type aliases ` using the :keyword:`type` statement, which creates an instance of :class:`~typing.TypeAliasType`:: @@ -306,7 +306,7 @@ Type aliases can also be :ref:`generic `:: type Point[T] = tuple[T, T] -The new syntax also allows declaring :class:`~typing.TypeVarTuple` +The new syntax allows declaring :class:`~typing.TypeVarTuple` and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.TypeVar` parameters with bounds or constraints:: @@ -316,14 +316,16 @@ parameters with bounds or constraints:: type IntOrStrSequence[T: (int, str)] = Sequence[T] # TypeVar with constraints The value of type aliases and the bound and constraints of type variables -created through this syntax are evaluated only on demand, enabling the use of -forward references to types defined later in the file. +created through this syntax are evaluated only on demand (see +:ref:`lazy-evaluation`). This means type aliases are able to refer to other +types defined later in the file. Type parameters declared through a type parameter list are visible within the scope of the declaration and any nested scopes, but not in the outer scope. For example, they can be used in the type annotations for the methods of a generic class or in the class body. However, they cannot be used in the module scope after -the class is defined. +the class is defined. See :ref:`type-params` for a detailed description of the +runtime semantics of type parameters. In order to support these scoping semantics, a new kind of scope is introduced, the :ref:`annotation scope `. Annotation scopes behave for the diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index b957980fb1abf5..caf73cfb7e926c 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -444,15 +444,13 @@ PyDoc_STRVAR(typevar_doc, "Type variable.\n\ \n\ The preferred way to construct a type variable is via the dedicated syntax\n\ -for :ref:`generic functions `,\n\ -:ref:`generic classes `, and\n\ -:ref:`generic type aliases `::\n\ +for generic functions, classes, and type aliases:\n\ \n\ class Sequence[T]: # T is a TypeVar\n\ ...\n\ \n\ This syntax can also be used to create bound and constrained type\n\ -variables::\n\ +variables:\n\ \n\ class StrSequence[S: str]: # S is a TypeVar bound to str\n\ ...\n\ @@ -462,7 +460,7 @@ variables::\n\ ...\n\ \n\ However, if desired, reusable type variables can also be constructed\n\ -manually, like so::\n\ +manually, like so:\n\ \n\ T = TypeVar('T') # Can be anything\n\ S = TypeVar('S', bound=str) # Can be any subtype of str\n\ @@ -473,8 +471,7 @@ checkers. They serve as the parameters for generic types as well\n\ as for generic function and type alias definitions.\n\ \n\ The variance of type variables is inferred by type checkers when they are created\n\ -through the type parameter syntax and when\n\ -``infer_variance=True`` is passed.\n\ +through the type parameter syntax and when ``infer_variance=True`` is passed.\n\ Manually created type variables may be explicitly marked covariant or\n\ contravariant by passing ``covariant=True`` or ``contravariant=True``.\n\ By default, manually created type variables are invariant. See PEP 484\n\ @@ -939,7 +936,14 @@ static PyMethodDef paramspec_methods[] = { PyDoc_STRVAR(paramspec_doc, "Parameter specification variable.\n\ \n\ -Usage::\n\ +The preferred way to construct a parameter specification is via the dedicated syntax\n\ +for generic functions, classes, and type aliases, where\n\ +the use of '**' creates a parameter specification:\n\ +\n\ + type IntFunc[**P] = Callable[P, int]\n\ +\n\ +For compatibility with Python 3.11 and earlier, ParamSpec objects\n\ +can also be created as follows:\n\ \n\ P = ParamSpec('P')\n\ \n\ @@ -949,12 +953,9 @@ callable to another callable, a pattern commonly found in higher order\n\ functions and decorators. They are only valid when used in ``Concatenate``,\n\ or as the first argument to ``Callable``, or as parameters for user-defined\n\ Generics. See class Generic for more information on generic types. An\n\ -example for annotating a decorator::\n\ -\n\ - T = TypeVar('T')\n\ - P = ParamSpec('P')\n\ +example for annotating a decorator:\n\ \n\ - def add_logging(f: Callable[P, T]) -> Callable[P, T]:\n\ + def add_logging[**P, T](f: Callable[P, T]) -> Callable[P, T]:\n\ '''A type-safe decorator to add logging to a function.'''\n\ def inner(*args: P.args, **kwargs: P.kwargs) -> T:\n\ logging.info(f'{f.__name__} was called')\n\ @@ -966,17 +967,9 @@ example for annotating a decorator::\n\ '''Add two numbers together.'''\n\ return x + y\n\ \n\ -Parameter specification variables defined with covariant=True or\n\ -contravariant=True can be used to declare covariant or contravariant\n\ -generic types. These keyword arguments are valid, but their actual semantics\n\ -are yet to be decided. See PEP 612 for details.\n\ -\n\ Parameter specification variables can be introspected. e.g.:\n\ \n\ P.__name__ == 'P'\n\ - P.__bound__ == None\n\ - P.__covariant__ == False\n\ - P.__contravariant__ == False\n\ \n\ Note that only parameter specification variables defined in global scope can\n\ be pickled.\n\ @@ -1172,9 +1165,18 @@ static PyMethodDef typevartuple_methods[] = { }; PyDoc_STRVAR(typevartuple_doc, -"Type variable tuple.\n\ +"Type variable tuple. A specialized form of type variable that enables\n\ +variadic generics.\n\ +\n\ +The preferred way to construct a type variable tuple is via the dedicated syntax\n\ +for generic functions, classes, and type aliases, where a single\n\ +'*' indicates a type variable tuple:\n\ +\n\ + def move_first_element_to_last[T, *Ts](tup: tuple[T, *Ts]) -> tuple[*Ts, T]:\n\ + return (*tup[1:], tup[0])\n\ \n\ -Usage:\n\ +For compatibility with Python 3.11 and earlier, TypeVarTuple objects\n\ +can also be created as follows:\n\ \n\ Ts = TypeVarTuple('Ts') # Can be given any name\n\ \n\ @@ -1438,6 +1440,23 @@ PyDoc_STRVAR(typealias_doc, Type aliases are created through the type statement:\n\ \n\ type Alias = int\n\ +\n\ +In this example, Alias and int will be treated equivalently by static\n\ +type checkers.\n\ +\n\ +At runtime, Alias is an instance of TypeAliasType. The __name__ attribute\n\ +holds the name of the type alias. The value of the type\n\ +alias is stored in the __value__ attribute. It is evaluated lazily, so\n\ +the value is computed only if the attribute is accessed.\n\ +\n\ +Type aliases can also be generic:\n\ +\n\ + type ListOrSet[T] = list[T] | set[T]\n\ +\n\ +In this case, the type parameters of the alias are stored in the\n\ +__type_params__ attribute.\n\ +\n\ +See PEP 695 for more information.\n\ "); static PyNumberMethods typealias_as_number = { @@ -1486,14 +1505,14 @@ PyDoc_STRVAR(generic_doc, \n\ A generic type is typically declared by inheriting from\n\ this class parameterized with one or more type variables.\n\ -For example, a generic mapping type might be defined as::\n\ +For example, a generic mapping type might be defined as:\n\ \n\ class Mapping(Generic[KT, VT]):\n\ def __getitem__(self, key: KT) -> VT:\n\ ...\n\ # Etc.\n\ \n\ -This class can then be used as follows::\n\ +This class can then be used as follows:\n\ \n\ def lookup_name(mapping: Mapping[KT, VT], key: KT, default: VT) -> VT:\n\ try:\n\ From 0cfeb9241985d7addffafc3efec642a6daaafda9 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 26 May 2023 08:52:00 -0700 Subject: [PATCH 53/56] More comments --- Doc/library/dis.rst | 4 ++-- Doc/library/typing.rst | 36 +++++++++++++------------------- Doc/reference/compound_stmts.rst | 4 ++-- Doc/reference/executionmodel.rst | 4 ++-- Doc/reference/simple_stmts.rst | 8 ++++--- 5 files changed, 25 insertions(+), 31 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index c135ebd70a73e3..cb9a4b1ee2d77d 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1270,11 +1270,11 @@ iterations of the loop. slot ``i`` of the "fast locals" storage in this mapping. If the name is not found there, loads it from the cell contained in slot ``i``, similar to :opcode:`LOAD_DEREF`. This is used for loading - free variables in class bodies and in + free variables in class bodies (which previously used + :opcode:`!LOAD_CLASSDEREF`) and in :ref:`annotation scopes ` within class bodies. .. versionadded:: 3.12 - Replaces the previous instruction ``LOAD_CLASSDEREF``. .. opcode:: STORE_DEREF (i) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 224035268cb9f9..a9ea4b9fee3ad4 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -316,7 +316,7 @@ Or by using the :class:`TypeVar` factory directly:: return l[0] .. versionchanged:: 3.12 - Syntactic support for generics is new in version 3.12. + Syntactic support for generics is new in Python 3.12. .. _user-defined-generics: @@ -1697,16 +1697,16 @@ without the dedicated syntax, as documented below. ``P.args`` and ``P.kwargs`` are instances respectively of :class:`ParamSpecArgs` and :class:`ParamSpecKwargs`. + .. attribute:: __name__ + + The name of the parameter specification. + Parameter specification variables created with ``covariant=True`` or ``contravariant=True`` can be used to declare covariant or contravariant generic types. The ``bound`` argument is also accepted, similar to :class:`TypeVar`. However the actual semantics of these keywords are yet to be decided. - .. attribute:: __name__ - - The name of the parameter specification. - .. versionadded:: 3.10 .. versionchanged:: 3.12 @@ -1755,9 +1755,7 @@ without the dedicated syntax, as documented below. .. attribute:: __name__ - The name of the type alias. - - Example:: + The name of the type alias:: >>> type Alias = int >>> Alias.__name__ @@ -1765,9 +1763,7 @@ without the dedicated syntax, as documented below. .. attribute:: __module__ - The module in which the type alias was defined. - - Example:: + The module in which the type alias was defined:: >>> type Alias = int >>> Alias.__module__ @@ -1776,11 +1772,9 @@ without the dedicated syntax, as documented below. .. attribute:: __type_params__ The type parameters of the type alias, or an empty tuple if the alias is - not generic. + not generic: - Example:: - - .. doctest:: + .. doctest:: >>> type ListOrSet[T] = list[T] | set[T] >>> ListOrSet.__type_params__ @@ -1791,13 +1785,11 @@ without the dedicated syntax, as documented below. .. attribute:: __value__ - The type alias's value. This is lazily evaluated, so names used in the - definition of the alias are not resolved until the ``__value__`` attribute - is accessed. - - Example:: + The type alias's value. This is :ref:`lazily evaluated `, + so names used in the definition of the alias are not resolved until the + ``__value__`` attribute is accessed: - .. doctest:: + .. doctest:: >>> type Mutually = Recursive >>> type Recursive = Mutually @@ -1869,7 +1861,7 @@ These are not used in annotations. They are building blocks for declaring types. key: T group: list[T] - # For creating a NamedTuple on Python 3.5 or lower + # A functional syntax is also supported Employee = NamedTuple('Employee', [('name', str), ('id', int)]) .. versionchanged:: 3.6 diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 412d3692e38a64..9b98a7155d7573 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1263,7 +1263,7 @@ the type parameters can be retrieved from the function's ``__type_params__`` attribute. See :ref:`generic-functions` for more. .. versionchanged:: 3.12 - Type parameters are new in version 3.12. + Type parameter lists are new in Python 3.12. .. index:: triple: default; parameter; value @@ -1450,7 +1450,7 @@ the type parameters can be retrieved from the class's ``__type_params__`` attribute. See :ref:`generic-classes` for more. .. versionchanged:: 3.12 - Type parameters are new in version 3.12. + Type parameter lists are new in Python 3.12. **Programmer's note:** Variables defined in the class definition are class attributes; they are shared by instances. Instance attributes can be set in a diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 038be37884b43f..cea3a56ba51644 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -180,7 +180,7 @@ However, the following will succeed:: class A: type Alias = Nested - class Nested: ... + class Nested: pass print(A.Alias.__value__) # @@ -225,7 +225,7 @@ Annotation scopes differ from function scopes in the following ways: syntactic elements that can appear within annotation scopes can introduce new names. * While annotation scopes have an internal name, that name is not reflected in the :term:`__qualname__ ` of objects defined within the scope. - Instead, the ``__qualname__`` + Instead, the :attr:`!__qualname__` of such objects is as if the object were defined in the enclosing scope. .. versionadded:: 3.12 diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index a250501483f18c..662a4b643c4378 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -1024,7 +1024,7 @@ The :keyword:`!type` statement .. productionlist:: python-grammar type_stmt: 'type' `identifier` [`type_params`] "=" `expression` -The :keyword:`!type` statement introduces a type alias, which is an instance +The :keyword:`!type` statement declares a type alias, which is an instance of :class:`typing.TypeAliasType`. For example, the following statement creates a type alias:: @@ -1038,10 +1038,12 @@ This code is roughly equivalent to:: Point = typing.TypeAliasType("Point", VALUE_OF_Point()) ``annotation-def`` indicates an :ref:`annotation scope `, which behaves -mostly like a function, but with several small differences. The value of the +mostly like a function, but with several small differences. + +The value of the type alias is evaluated in the annotation scope. It is not evaluated when the type alias is created, but only when the value is accessed through the type alias's -``__value__`` attribute (see :ref:`lazy-evaluation`). +:attr:`!__value__` attribute (see :ref:`lazy-evaluation`). This allows the type alias to refer to names that are not yet defined. Type aliases may be made generic by adding a :ref:`type parameter list ` From f978766e122c04353e6efc17916bd28f1b20aa68 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 26 May 2023 09:17:23 -0700 Subject: [PATCH 54/56] Docstring fixes --- Objects/typevarobject.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index caf73cfb7e926c..1956efd1188a6b 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -446,18 +446,18 @@ PyDoc_STRVAR(typevar_doc, The preferred way to construct a type variable is via the dedicated syntax\n\ for generic functions, classes, and type aliases:\n\ \n\ - class Sequence[T]: # T is a TypeVar\n\ - ...\n\ + class Sequence[T]: # T is a TypeVar\n\ + ...\n\ \n\ This syntax can also be used to create bound and constrained type\n\ variables:\n\ \n\ - class StrSequence[S: str]: # S is a TypeVar bound to str\n\ - ...\n\ + class StrSequence[S: str]: # S is a TypeVar bound to str\n\ + ...\n\ \n\ \n\ - class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes\n\ - ...\n\ + class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes\n\ + ...\n\ \n\ However, if desired, reusable type variables can also be constructed\n\ manually, like so:\n\ @@ -1184,7 +1184,7 @@ Just as a TypeVar (type variable) is a placeholder for a single type,\n\ a TypeVarTuple is a placeholder for an *arbitrary* number of types. For\n\ example, if we define a generic class using a TypeVarTuple:\n\ \n\ - class C(Generic[*Ts]): ...\n\ + class C[*Ts]: ...\n\ \n\ Then we can parameterize that class with an arbitrary number of type\n\ arguments:\n\ From bba092d28bada4a4dd62be2bb9866af4d1c7102b Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 26 May 2023 09:31:58 -0700 Subject: [PATCH 55/56] tables --- Doc/library/dis.rst | 84 +++++++++++++++++++++++++++++++-------------- 1 file changed, 58 insertions(+), 26 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index cb9a4b1ee2d77d..9b90f1ef23d92c 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1526,23 +1526,45 @@ iterations of the loop. The operand determines which intrinsic function is called: - * ``INTRINSIC_1_INVALID`` Not valid - * ``INTRINSIC_PRINT`` Prints the argument to standard out. - Used in the REPL. - * ``INTRINSIC_IMPORT_STAR`` Performs ``import *`` for the named module. - * ``INTRINSIC_STOPITERATION_ERROR`` Extracts the return value from - a ``StopIteration`` exception. - * ``INTRINSIC_ASYNC_GEN_WRAP`` Wraps an aync generator value - * ``INTRINSIC_UNARY_POSITIVE`` Performs the unary ``+`` operation - * ``INTRINSIC_LIST_TO_TUPLE`` Converts a list to a tuple - * ``INTRINSIC_TYPEVAR`` Creates a :class:`typing.TypeVar` - * ``INTRINSIC_PARAMSPEC`` Creates a :class:`typing.ParamSpec` - * ``INTRINSIC_TYPEVARTUPLE`` Creates a :class:`typing.TypeVarTuple` - * ``INTRINSIC_SUBSCRIPT_GENERIC`` Returns :class:`typing.Generic` - subscripted with the value on top of the stack. - * ``INTRINSIC_TYPEALIAS`` Creates a :class:`typing.TypeAliasType`; - used in the :keyword:`type` statement. The argument is a tuple of - the type alias's name, type parameters, and value. + +-----------------------------------+-----------------------------------+ + | Operand | Description | + +===================================+===================================+ + | ``INTRINSIC_1_INVALID`` | Not valid | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_PRINT`` | Prints the argument to standard | + | | out. Used in the REPL. | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_IMPORT_STAR`` | Performs ``import *`` for the | + | | named module. | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_STOPITERATION_ERROR`` | Extracts the return value from a | + | | ``StopIteration`` exception. | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_ASYNC_GEN_WRAP`` | Wraps an aync generator value | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_UNARY_POSITIVE`` | Performs the unary ``+`` | + | | operation | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_LIST_TO_TUPLE`` | Converts a list to a tuple | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_TYPEVAR`` | Creates a :class:`typing.TypeVar` | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_PARAMSPEC`` | Creates a | + | | :class:`typing.ParamSpec` | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_TYPEVARTUPLE`` | Creates a | + | | :class:`typing.TypeVarTuple` | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_SUBSCRIPT_GENERIC`` | Returns :class:`typing.Generic` | + | | subscripted with the argument | + +-----------------------------------+-----------------------------------+ + | ``INTRINSIC_TYPEALIAS`` | Creates a | + | | :class:`typing.TypeAliasType`; | + | | used in the :keyword:`type` | + | | statement. The argument is a tuple| + | | of the type alias's name, | + | | type parameters, and value. | + +-----------------------------------+-----------------------------------+ .. versionadded:: 3.12 @@ -1554,15 +1576,25 @@ iterations of the loop. The operand determines which intrinsic function is called: - * ``INTRINSIC_2_INVALID`` Not valid - * ``INTRINSIC_PREP_RERAISE_STAR`` Calculates the :exc:`ExceptionGroup` - to raise from a ``try-except*``. - * ``INTRINSIC_TYPEVAR_WITH_BOUND`` Creates a :class:`typing.TypeVar` - with a bound. - * ``INTRINSIC_TYPEVAR_WITH_CONSTRAINTS`` Creates a :class:`typing.TypeVar` - with constraints. - * ``INTRINSIC_SET_FUNCTION_TYPE_PARAMS`` Sets the ``__type_params__`` - attribute of a function. + +----------------------------------------+-----------------------------------+ + | Operand | Description | + +========================================+===================================+ + | ``INTRINSIC_2_INVALID`` | Not valid | + +----------------------------------------+-----------------------------------+ + | ``INTRINSIC_PREP_RERAISE_STAR`` | Calculates the | + | | :exc:`ExceptionGroup` to raise | + | | from a ``try-except*``. | + +----------------------------------------+-----------------------------------+ + | ``INTRINSIC_TYPEVAR_WITH_BOUND`` | Creates a :class:`typing.TypeVar` | + | | with a bound. | + +----------------------------------------+-----------------------------------+ + | ``INTRINSIC_TYPEVAR_WITH_CONSTRAINTS`` | Creates a | + | | :class:`typing.TypeVar` with | + | | constraints. | + +----------------------------------------+-----------------------------------+ + | ``INTRINSIC_SET_FUNCTION_TYPE_PARAMS`` | Sets the ``__type_params__`` | + | | attribute of a function. | + +----------------------------------------+-----------------------------------+ .. versionadded:: 3.12 From 57ddbef36d7d7e91e6b982ef50efee9d6e1e08c9 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 26 May 2023 09:48:23 -0700 Subject: [PATCH 56/56] more fixes --- Doc/reference/compound_stmts.rst | 2 +- Objects/typevarobject.c | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 9b98a7155d7573..6d30eccab1990f 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1657,7 +1657,7 @@ with more precision. The scope of type parameters is modeled with a special function (technically, an :ref:`annotation scope `) that wraps the creation of the generic object. -Generic functions, classes, and type aliases have a :attr:`__type_params__` +Generic functions, classes, and type aliases have a :attr:`!__type_params__` attribute listing their type parameters. Type parameters come in three kinds: diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 1956efd1188a6b..0b7d84c706d94e 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -454,7 +454,6 @@ variables:\n\ \n\ class StrSequence[S: str]: # S is a TypeVar bound to str\n\ ...\n\ -\n\ \n\ class StrOrBytesSequence[A: (str, bytes)]: # A is a TypeVar constrained to str or bytes\n\ ...\n\