Generate type stubs at the same time as generating modules

[junkmd]

Abstract of proposal

I wish client.GetModule generates Friendly.pyi type stub at same time as generating _xxxxxxxxxxxx_xxxxxxxx_xxxxxxxx_xxxxxxxxxxxx_x_x_x.py and Friendly.py runtime modules.

Rationale

comtypes dynamically creates type-defined modules. This is a feature that other COM manipulation libraries don't have.

However, the type definitions are imported from the wrapper module _xxxxxxxxxxxx_xxxxxxxx_xxxxxxxx_xxxxxxxxxxxx_x_x_x.py to the user-friendly module Friendly.py as shown below, so all type information is hidden from the type checker.

from comtypes.gen import _xxxxxxxxxxxx_xxxxxxxx_xxxxxxxx_xxxxxxxxxxxx_x_x_x
globals().update(_xxxxxxxxxxxx_xxxxxxxx_xxxxxxxx_xxxxxxxxxxxx_x_x_x.__dict__)
__name__ = 'comtypes.gen.Friendly'

In terms of coding usability, it is no different than dynamically calling methods and attributes from a Dispatch object, and it is difficult to figure out what API the module has.

Also, the methods and properties of the derived classes of IUnknown and CoClass in the generated module are defined by the metaclass, so the type checker cannot obtain information on arguments and return values.

I would like to add the feature to generate type stubs in a format according to PEP561 at the same time as the runtime module to client.GetModule to figure out easily what API the module has.

When

• After the next version of 1.1.12 is released.

and

• After the refactoring of modules that defines the base class that is imported into the generated module is finished.

and

• After the providing type stubs or adding type hint comments in a format according to PEP484 into some static modules below.
  • comtypes/__init__.py
  • comtypes/automation.py
  • comtypes/client/__init__.py
  • comtypes/client/_genetate.py

[junkmd]

Memo for type hint symbols

Question: Does it require backward compatibility with generic-alias for builtin classes, related to PEP585?

Answer: No, "There are currently no plans to remove the aliases from typing."

see below;
python/typing#1230
https://discuss.python.org/t/concern-about-pep-585-removals/15901
microsoft/pylance-release#3066

[junkmd]

Memo for type variable

IUnknown.QueryInterface is useful for casting

from typing import Optional
from typing import Type  # depreciated in >= Py3.9, see PEP585.
from typing import TypeVar

_T_IUnknown = TypeVar("_T_IUnknown", bound=IUnknown)

class IUnknown(c_void_p):
    def AddRef(self) -> int: ...
    def QueryInterface(self, interface: Type[_T_IUnknown], iid: Optional[GUID] = ...) -> _T_IUnknown: ...
    def Release(self) -> int: ...

[junkmd]

Memo for Pointer policies

If def somefunc() returns instance that type is POINTER(IUnknown) in runtime, it will be def somefunc() -> IUnknown

Because it is not a lie. And good way to know what APIs the class has.

• isinstance(CreateObject("Scripting.Dictionary"), POINTER(Scripting.IDictionary)) returns True,
• isinstance(CreateObject("Scripting.Dictionary"), Scripting.IDictionary) returns True.
• The type checker does not know what methods and attributes it has from POINTER(IUnknown). Most developers will want the IUnknown interface.
  • ctypes.pointer behaves like generics with type stubs, even though it is not generics at runtime.
    - I think this is probabaly because it can be easily expressed "the value of a pointer is c_int" as pointer[c_int].
    - However, this is possible because ctypes definitions only have like c_int those have interfaces with no different from _SimpleCData.
    - It is not assumed that the object such as comtypes.IDispatch can have more than the methods and attributes that _SimpleCData has.

[junkmd]

• ctypes.pointer behaves like generics with type stubs, even though it is not generics at runtime.

Currently, ctypes.pointer is defined as function in the type stub as well as in the implementation.
ctypes.pointer(_CT) returns ctypes._Pointer[_CT].
see python/typeshed#8446 and code snippet.

[junkmd]

I thought what to do about the problem of COMError and CopyComPointer being Unknown because there is no type stub for _ctypes.

I posted an issue to typeshed for adding new stub file(python/typeshed#8571).

I got agreement to add stdlib/_ctypes.pyi, so I submitted a PR(python/typeshed#8582) and it was merged.

[junkmd]

The Array was also Unknown because it was imported from _ctypes.

I will submit a PR to change the Array import from _ctypes to import from ctypes since both are the same stuff.

comtypes/comtypes/automation.py

Line 556 in cc9a013

from _ctypes import Array as _CArrayType

[junkmd]

PEP585 will be updated.

see python/peps#2778

[junkmd]

Type hinting for statically defined module, like comtypes/__init__.py.

I realized that it would be hard to update the .pyi files statically defined to match the updating .py files statically defined in same time as well.

So I considered writing type annotations in the .py files so that they would not affect them at runtime so that they would be compatible with the older versions.

It is able to write type annotations in according to PEP484's "placed in a # type: comment".

I tried static type checking by mypy and pyright(pylance) in VSCode, Py38 env.

The result is below.

a = ... # type: A | B(runtime available in Py3.10) works for both, but list[Any] and (A | B) -> None raise error by mypy.

Therefore, we must use generics like typing.List instead of builitins.list, even if it is annotations in comments.
Adding only-type-checking symbols are easily by tying.TYPE_CHECKING.
So we don't need afraid that unexpected symbols are added in runtime.

if sys.version_info >= (3, 5):
    from typing import TYPE_CHECKING
else:
    TYPE_CHECKING = False

if TYPE_CHECKING:
    from typing import List, Tuple

a = []  # type: List[Tuple[str, int]]

(see https://peps.python.org/pep-0484/#runtime-or-type-checking)

[junkmd]

I would like to use a similar process to generate runtime code and type stub code.

I will create a base class that abstracts the tools.codegenerator.CodeGenerator class.
I would like to implement a concrete class for runtime code generation and a concrete class for type stub code generation inherited from that base class.

However, defining these abstract and concrete classes in the same tools.codegenerator will result in complicated class names and bloated code in a single module.

Therefore, tools/codegenerator.py should be separated into some module files.

[junkmd]

@kdschlosser

modules dictionary

The Friendly.py issue is also covered in #328. It would be great if it were easy to stop manipulating the module dictionary.

a really large assortment of them already coded up here #256

This is a very nice suggestion.

PROPVARIANT

I'm pretty sure you mean #263.

My concern is that if all of these are included at once in one PR, it may be difficult to review.
It would be helpful if you could make the PR as small as possible, as close to a commit unit as possible, or in the way recommended by google's Small CLs.
And need to be aware of testing.

Please share in this issue about the overall changes.

Thank you for taking the time to read this long article.

[kdschlosser]

No I don't mean #263.

I did a redo of the code and made it easy to access the data.

I did want to point out then when doing either inline type hinting or the commented way this is how you should go about doing it.

This is from comtypes.typeinfo

class ITypeLib(IUnknown):
    _iid_: GUID = GUID("{00020402-0000-0000-C000-000000000046}")

    # type-checking only methods use the default implementation that comtypes
    # automatically creates for COM methods.

    def GetTypeInfoCount(self) -> int:
        """Return the number of type informations"""
        return self._GetTypeInfoCount()

    def GetTypeInfo(self, index: int) -> "ITypeInfo":
        """Load type info by index"""
        return self._GetTypeInfo(index)

    def GetTypeInfoType(self, index: int) -> TYPEKIND:
        """Return the TYPEKIND of type information"""
        return self._GetTypeInfoType(index)

    def GetTypeInfoOfGuid(self, guid: GUID) -> "ITypeInfo":
        """Return type information for a guid"""
        return self._GetTypeInfoOfGuid(guid)

    def GetTypeComp(self) -> "ITypeComp":
        """Return an ITypeComp pointer."""
        return self._GetTypeComp()

    def GetDocumentation(
        self,
        index: int
    ) -> Tuple[str, str, int, Optional[str]]:
        """Return documentation for a type description."""
        return self._GetDocumentation(index)

    def ReleaseTLibAttr(self, ptla: "TLIBATTR") -> None:
        """Release TLIBATTR"""
        self._ReleaseTLibAttr(byref(ptla))

    def GetLibAttr(self) -> "TLIBATTR":
        """Return type library attributes"""
        return _deref_with_release(self._GetLibAttr(), self.ReleaseTLibAttr)

    def IsName(self, name: str, lHashVal: Optional[int] = 0) -> Optional[str]:
        """Check if there is type information for this name.

        Returns the name with capitalization found in the type
        library, or None.
        """
        from ctypes import create_unicode_buffer
        namebuf = create_unicode_buffer(name)
        found = BOOL()
        self.__com_IsName(namebuf, lHashVal, byref(found))
        if found.value:
            return namebuf[:].split("\0", 1)[0]
        return None

    def FindName(
        self,
        name: str,
        lHashVal: Optional[int] = 0
    ) -> Optional[Tuple[int, "ITypeInfo"]]:
        # Hm...
        # Could search for more than one name - should we support this?
        found = c_ushort(1)
        tinfo = POINTER(ITypeInfo)()
        memid = MEMBERID()
        self.__com_FindName(
            name,
            lHashVal,
            byref(tinfo),
            byref(memid),
            byref(found)
        )

        if found.value:
            return memid.value, tinfo  # type: ignore
        return None

If memory serves I believe this is how the COM interfaces are set up.

you have the ability to set methods using the same name as the underlying COM method. You can access the method so that the "in" and "out" flags work properly by calling the method with a preceding underscore. you can also access the COM method by preceding the method name with "com", when going this route the "in" and "out" flags are ignored and you have to pass the data containers as is needed.

Structures and Unions from ctypes are set up a bit differently

class N11tagTYPEDESC5DOLLAR_203E(Union):
    # C:/Programme/gccxml/bin/Vc71/PlatformSDK/oaidl.h 584

    @property
    def lptdesc(self) -> TYPEDESC:
        return TYPEDESC()
        
    @lptdesc.setter
    def lptdesc(self, value: TYPEDESC):
        pass

    @property
    def lpadesc(self) -> tagARRAYDESC:
        return tagARRAYDESC()

    @lpadesc.setter
    def lpadesc(self, value: tagARRAYDESC):
        pass

    @property
    def hreftype(self) -> int:
        return int()

    @hreftype.setter
    def hreftype(self, value: int):
        pass

    _fields_ = [
        # C:/Programme/gccxml/bin/Vc71/PlatformSDK/oaidl.h 584
        ('lptdesc', POINTER(tagTYPEDESC)),
        ('lpadesc', POINTER(tagARRAYDESC)),
        ('hreftype', HREFTYPE),
    ]

I know that looks a little goofy but what happens on the back end is when the class gets built the properties get overwritten by what is in fields

An IDE does not see the c code that works that magic so it is a seamless transition.

Using the mechanism you are using makes the code more difficult to read. A comment can be added to the methods that get overridden saying that they get overridden by what is in fields

I would really consider using the data_types module I have written. It will make the generated typelibs match what is actually in the typelib. I know that C code really has no difference between things like a LONG and an INT or a HANDLE and a HWND, it would still be nice to have the proper data type being seen in the generated code.

I also want to change up the data_types file so that instead of a variable being created for a specific data type a subclass of the data type is used instead.

I have done this for enumerations and I have cleverly come up with a way to wrap an enumeration item in a manner that allows it to be identified by its name or by its value.

[kdschlosser]

You also have some odd type hinting.
Here is the type hinting you have set up for a pointer to IUnknown

Type[_Pointer[IUnknown]]]

This would be easier

    from ctypes import POINTER
    from typing import TypeVar, Type

    _TV_POINTER = TypeVar("_TV_POINTER", bound=POINTER)
    _T_POINTER = Type[_TV_POINTER]

and that allows your type hint to be _T_POINTER[IUnknown] without Python pitching a fit when the code is run

The use of TYPE_CHECKING really should only be reserved for the importation of modules that would cause a circular import. You would need to wrap whatever the type hint is that is in the module with double quotes in order for it to work properly.

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from some_module import SomeClass

def do() -> "SomeClass":
    ...

[kdschlosser]

Those suggestions make the code a lot easier to read. They are only suggestions and you can do whatever it is that you like.

You should have the maintainer of comtypes create a new branch and label that branch so it aligns with a milestone/project that has been made for dropping support for Python 3. This allows modifications to be made now without effecting the master branch and it also allows the code to get used and tested.

[junkmd]

@kdschlosser

No I don't mean #263.

I did a redo of the code and made it easy to access the data.

If so, please PR that revised version.
Regardless of type hinting, it should be a useful addition to comtypes.

This is from comtypes.typeinfo

This is certainly a good way to go.

The method defined by the metaclass is annotated with Callable, the argument names that are available in runtime are lost from type information.

If memory serves I believe this is how the COM interfaces are set up.

you have the ability to set methods using the same name as the underlying COM method. You can access the method so that the "in" and "out" flags work properly by calling the method with a preceding underscore. you can also access the COM method by preceding the method name with "com", when going this route the "in" and "out" flags are ignored and you have to pass the data containers as is needed.

I understood this behavior when I refactored the metaclass that generates methods(#367, #368 and #373).

Structures and Unions from ctypes are set up a bit differently

I have some opinions on this.
I think would be the way to work for Python3.

class N11tagTYPEDESC5DOLLAR_203E(Union):
    lptdesc: TYPEDESC
    ...
    _fields_ = [
        ('lptdesc', POINTER(tagTYPEDESC)),
        ...
    ]

I don't want to have something recognized as a property that is not a property in runtime.
I think this way would be a good fit since it is truly an annotation to a "instance variable".

I would really consider using the data_types module I have written. It will make the generated typelibs match what is actually in the typelib. I know that C code really has no difference between things like a LONG and an INT or a HANDLE and a HWND, it would still be nice to have the proper data type being seen in the generated code.

I also want to change up the data_types file so that instead of a variable being created for a specific data type a subclass of the data type is used instead.

I have done this for enumerations and I have cleverly come up with a way to wrap an enumeration item in a manner that allows it to be identified by its name or by its value.

This should be really great.

I would like you to implement this feature in a different issue scope than this, regardless of type hinting.

[junkmd]

@kdschlosser

    from ctypes import POINTER
    from typing import TypeVar, Type

    _TV_POINTER = TypeVar("_TV_POINTER", bound=POINTER)
    _T_POINTER = Type[_TV_POINTER]

This should be a useful generics. Thanks for the advice.

[junkmd]

@kdschlosser

The use of TYPE_CHECKING really should only be reserved for the importation of modules that would cause a circular import. You would need to wrap whatever the type hint is that is in the module with double quotes in order for it to work properly.

Previously, inadvertently adding/existing symbols in a module would cause a bug when a COM object with the same name existed on the type library(This caused #330).

Currently, I have set __known_symbols__ for some modules, so adding symbols for type hints is not likely to cause any problems(see #360).

So when we drop the Python 2 system and no longer need the workaround, we should use the way you suggested.

[kdschlosser]

I feel that making a branch to start working on dropping python2 is the way to go. There is really no sense in doing all the type hints for python 2 and 3 just to have to change them all in a year from now.

[junkmd]

@kdschlosser

I feel that making a branch to start working on dropping python2 is the way to go. There is really no sense in doing all the type hints for python 2 and 3 just to have to change them all in a year from now.

I agree.

So I will create a new branch once I get agreement from the other maintainers.

If we can do that, then let's proceed with the type hinting in there.

[junkmd]

@vasily-v-ryabov
@cfarrow
@jaraco

@kdschlosser knows how to do better with inline annotations for this type hinting enhancement without generating pyi files.

This is a feature that is not available with the current Python2 support.

So I would like to create a separate drop_py2 branch and plan to merge that into master when Python2 is removed from support in the near future.

If there are no objections in the next week, or if you agree, we will proceed in that direction.

Please consider this.

[junkmd]

@kdschlosser

I sent mention to the other maintainers.

While waiting for replies, please PR any features you would like to merge into the current master.

I will review them.

[junkmd]

I created #392 to discuss regarding Python2 drops.

[kdschlosser]

I have some opinions on this. I think would be the way to work for Python3.

class N11tagTYPEDESC5DOLLAR_203E(Union):
    lptdesc: TYPEDESC
    ...
    _fields_ = [
        ('lptdesc', POINTER(tagTYPEDESC)),
        ...
    ]

I don't want to have something recognized as a property that is not a property in runtime. I think this way would be a good fit since it is truly an annotation to a "instance variable".

And my rebuttal to this is 2 fold. First is it is a property that gets created (see below) and second is docstrings!! While I know that docstrings can be added to attributes they are only for sphinx and not a built in python feature. They might show up in an IDE if the IDE supports sphinx style attribute docstrings. Doing it the way I have suggested is 100% working with all IDEs that support pyhton because properties are a built in feature of python and not sphinx.

https://github.com/python/cpython/blob/main/Modules/_ctypes/stgdict.c

line 585.

 if (isStruct) {
            prop = PyCField_FromDesc(desc, i,
                                   &field_size, bitsize, &bitofs,
                                   &size, &offset, &align,
                                   pack, big_endian);

and then in https://github.com/python/cpython/blob/main/Modules/_ctypes/cfield.c

line 208

static int
PyCField_set(CFieldObject *self, PyObject *inst, PyObject *value)
{
    CDataObject *dst;
    char *ptr;
    if (!CDataObject_Check(inst)) {
        PyErr_SetString(PyExc_TypeError,
                        "not a ctype instance");
        return -1;
    }
    dst = (CDataObject *)inst;
    ptr = dst->b_ptr + self->offset;
    if (value == NULL) {
        PyErr_SetString(PyExc_TypeError,
                        "can't delete attribute");
        return -1;
    }
    return PyCData_set(inst, self->proto, self->setfunc, value,
                     self->index, self->size, ptr);
}

static PyObject *
PyCField_get(CFieldObject *self, PyObject *inst, PyTypeObject *type)
{
    CDataObject *src;
    if (inst == NULL) {
        return Py_NewRef(self);
    }
    if (!CDataObject_Check(inst)) {
        PyErr_SetString(PyExc_TypeError,
                        "not a ctype instance");
        return NULL;
    }
    src = (CDataObject *)inst;
    return PyCData_get(self->proto, self->getfunc, inst,
                     self->index, self->size, src->b_ptr + self->offset);
}

Look at the function names PyCField_set and PyCField_get

It is a property that is created.

Now unfortunately there is no mechanics in place for docstrings and the properties set in place that get overridden by the fields happens when the class is built so for purposes of sphinx the docstrings would be useless and at runtime help could not be used but.. for the purposes of an IDE those docstrings do get displayed.

[kdschlosser]

gotta go to the backend mechanics to see what is actually happening.

Personally I like how comtypes works and how a method doesn't get overridden by what is in _methods_ and instead an alternative attribute gets created. This allows for special handling of that method to be performed. With ctypes Unions and Structures this is not the case and to be honest it frankly sucks because whatever data gets passed to a specific field you might want to alter/change. An example of this would be VARIANTBOOL which uses -1 for True and 0 for False so if a field using that data type you have to know to pass a -1 to it instead of passing True. This could be handled properly if a property could be created without it being overridden during the creation of the class. Not being able to pass True kind of removes the simplicity of use aspect. With respect to VARIANTBOOL I did find a way around this on the "get" side of things by overriding the value get/set descriptor with my own versions. I have no way to handle the set aspect because the value get/set descriptor isn't used in the backend to set the field. I have not dug into how it is done to see if it is even possible to override it. I am sure that I could override the fields that are created for a structure or union but only after the structure/union class has been created and that makes for some pretty ugly code.

[junkmd]

@kdschlosser

First is it is a property that gets created (see below) and second is docstrings!!

Thank you for the great commentary!
I respect your knowledge of cpython implementation.

>>> import ctypes
>>> class Foo(ctypes.Structure):
...     pass
... 
>>> Foo._fields_ = [("ham", ctypes.c_int)] 
>>> Foo.ham
<Field type=c_long, ofs=0, size=4>
>>> isinstance(Foo.ham, property) 
False

From the above, I was concerned that the Structure field was not a builtins.property, but I did not check until the implementation in cpython. Thanks for pointing this out.

>>> Foo.ham.__get__ 
<method-wrapper '__get__' of _ctypes.CField object at 0x0000017E53D8D3C0>
>>> Foo.ham.__set__ 
<method-wrapper '__set__' of _ctypes.CField object at 0x0000017E53D8D3C0>

Since both Field and builtins.property are data-descriptors and property is a "special-cases" unlike other custom descriptors, decorating with property seems to be in order.

As for docstring, I think it is inevitable that it will be lost in runtime. But

the purposes of an IDE those docstrings do get displayed

is helpful for developers.

[kdschlosser]

yessir. That is why these kinds of brain storming sessions are needed. we want to make sure the code is going to be right and that it is going to work properly.

The back end c code for ctypes is hard to follow. It takes a while to track down what exactly is going on. and if you call type on the class method that is set for the field the type of CField gets returned.

There technically speaking is a difference between a property and a get/set descriptor tho they do function almost identical. A property is just a convenience class for a get/set descriptor and it implements mechanisms to be used as decorators. It also provides some additional functions instead of calling __get__ and __set__ and __delete__. Those methods are fget, fset and fdel.

I am willing to bet if you did Foo.ham.fget you would probably end up with an attribute error. That means that the CField class is not a subclass of Property and is it's own class that implements __get__ and __set__. for the purposes of type hinting and docstrings they will serve the purpose.

[junkmd]

@kdschlosser

As property discussion reminds me, comtypes has the custom descriptors, named_property and bound_named_property.

I had almost written type hints for these, but deliberately avoided starting on them because they would be too complicated.

I may be wrong in some areas, but this is what I imagined.

from typing import (
    Any, Callable, Generic, NoReturn, Optional, overload, SupportsIndex,
    TypeVar
)


_R_Fget = TypeVar("_R_Fget")
_T_Instance = TypeVar("_T_Instance")


class bound_named_property(Generic[_R_Fget, _T_Instance]):
    def __init__(
        self,
        name: str,
        fget: Optional[Callable[..., _R_Fget]],
        fset: Optional[Callable[..., Any]],
        instance: _T_Instance
    ) -> None: ...
    def __getitem__(self, index: SupportsIndex) -> _R_Fget: ...
    def __call__(self, *args: Any) -> _R_Fget: ...
    def __setitem__(self, index: SupportsIndex, value: Any) -> None: ...
    def __iter__(self) -> NoReturn: ...


class named_property(Generic[_R_Fget, _T_Instance]):
    def __init__(
        self,
        name: str
        fget: Optional[Callable[..., _R_Fget]] = ...,
        fset: Optional[Callable[..., Any]] = ...,
        doc: Optional[text_type] = ...
    ) -> None: ...

    @overload
    def __get__(self, instance: _T_Instance, owner: Optional[_T_Instance] = ...) -> bound_named_property[_R_Fget, _T_Instance]: ...
    @overload
    def __get__(self, instance: None, owner: Optional[_T_Instance] = ...) -> named_property[_R_Fget, None]: ...

However, this would lose the type information of the arguments, so I considered a pattern using ParamSpec, which was introduced in Python 3.10.

# It's PEP585 and PEP604 style. Please think them will be changed to `typing` symbols.
_T = TypeVar("_T")
_R = TypeVar("_R")
_T_Inst = TypeVar("_T_Inst")
_GetP = ParamSpec("_GetP")
_SetP = ParamSpec("_SetP")

class bound_named_property(Generic[_T_Inst, _GetP, _R, _SetP]):
    name: str
    instance: _T_Inst
    fget: Callable[Concatenate[_T_Inst, _GetP], _R]
    fset: Callable[Concatenate[_T_Inst, _SetP], None]
    def __init__(self, name: str, fget: Callable[Concatenate[_T_Inst, _GetP], _R], fset: Callable[Concatenate[_T_Inst, _SetP], None], instance: _T_Inst) -> None: ...
    def __getitem__(self, index: Any) -> _R: ...
    def __call__(self, *args: _GetP.args, **kwargs: _GetP.kwargs) -> _R: ...
    def __setitem__(self, index: Any, value: Any) -> None: ...
    def __iter__(self) -> NoReturn: ...

class named_property(Generic[_T_Inst, _GetP, _R, _SetP]):
    name: str
    fget: None | Callable[Concatenate[_T_Inst, _GetP], _R]
    fset: None | Callable[Concatenate[_T_Inst, _SetP], None]
    __doc__: None | str
    def __init__(self, name: str, fget: Callable[Concatenate[_T_Inst, _GetP], _R] | None = ..., fset: Callable[Concatenate[_T_Inst, _SetP], None] | None = ..., doc: str | None = ...) -> None: ...
    @overload
    def __get__(self, instance: None, owner: type[_T_Inst]) -> named_property[None, _GetP, _R, _SetP]: ...
    @overload
    def __get__(self, instance: _T_Inst, owner: type[_T_Inst] | None) -> bound_named_property[_T_Inst, _GetP, _R, _SetP]: ...
    def __set__(self, instance: _T_Inst, value: Any) -> NoReturn: ...

Please let me know what you think about named_property and bound_named_property.

[kdschlosser]

type hinting those classes are pointless. actually type hinting most of what is in that file is pointless. This is because it is never really going to end up getting used at all. All of the stuff in the _memberspec file is for the dynamic creation of the methods and properties that are used when a class is dynamically built. Those type hints will never be realized when the user creates a COM interface. There is nothing that can be done for the users code and they would be responsible for the type hinting of their code.

It's because of the dynamic nature of how it works.

Technically speaking those classes should not be public classes either and they should be prefixed with an "_"..

[junkmd]

@kdschlosser

I see, so there is a better way to express the runtime behavior of named_property as type hinting.

And I also think _memberspec module implementation needs some refactoring futhermore.

[kdschlosser]

The back end mechanics of comtypes was designed for early Python 2. A lot has changed since then and I am sure that there is a lot of refactoring of the code that can be done. I don't fully grasp why there is like a line of classes used to set a simple property... But I am sure it probably had something to do with getting comtypes to work with early versions of Python 2.

[kdschlosser]

At some point I will dive into it in depth and see what is actually happening with the back end.

[junkmd]

@kdschlosser

I am curious about the methods of each class patched by _cominterface_meta._make_specials, too.

comtypes/comtypes/__init__.py

Lines 407 to 503 in 7e90e54

	def _make_specials(self):
	# This call installs methods that forward the Python protocols
	# to COM protocols.
	def has_name(name):
	# Determine whether a property or method named 'name'
	# exists
	if self._case_insensitive_:
	return name.lower() in self.__map_case__
	return hasattr(self, name)
	# XXX These special methods should be generated by the code generator.
	if has_name("Count"):
	@patcher.Patch(self)
	class _(object):
	def __len__(self):
	"Return the the 'self.Count' property."
	return self.Count
	if has_name("Item"):
	@patcher.Patch(self)
	class _(object):
	# 'Item' is the 'default' value. Make it available by
	# calling the instance (Not sure this makes sense, but
	# win32com does this also).
	def __call__(self, *args, **kw):
	"Return 'self.Item(*args, **kw)'"
	return self.Item(*args, **kw)
	# does this make sense? It seems that all standard typelibs I've
	# seen so far that support .Item also support ._NewEnum
	@patcher.no_replace
	def __getitem__(self, index):
	"Return 'self.Item(index)'"
	# Handle tuples and all-slice
	if isinstance(index, tuple):
	args = index
	elif index == _all_slice:
	args = ()
	else:
	args = (index,)
	try:
	result = self.Item(*args)
	except COMError as err:
	(hresult, text, details) = err.args
	if hresult == -2147352565: # DISP_E_BADINDEX
	raise IndexError("invalid index")
	else:
	raise
	# Note that result may be NULL COM pointer. There is no way
	# to interpret this properly, so it is returned as-is.
	# Hm, should we call __ctypes_from_outparam__ on the
	# result?
	return result
	@patcher.no_replace
	def __setitem__(self, index, value):
	"Attempt 'self.Item[index] = value'"
	try:
	self.Item[index] = value
	except COMError as err:
	(hresult, text, details) = err.args
	if hresult == -2147352565: # DISP_E_BADINDEX
	raise IndexError("invalid index")
	else:
	raise
	except TypeError:
	msg = "%r object does not support item assignment"
	raise TypeError(msg % type(self))
	if has_name("_NewEnum"):
	@patcher.Patch(self)
	class _(object):
	def __iter__(self):
	"Return an iterator over the _NewEnum collection."
	# This method returns a pointer to _some_ _NewEnum interface.
	# It relies on the fact that the code generator creates next()
	# methods for them automatically.
	#
	# Better would maybe to return an object that
	# implements the Python iterator protocol, and
	# forwards the calls to the COM interface.
	enum = self._NewEnum
	if isinstance(enum, types.MethodType):
	# _NewEnum should be a propget property, with dispid -4.
	#
	# Sometimes, however, it is a method.
	enum = enum()
	if hasattr(enum, "Next"):
	return enum
	# _NewEnum returns an IUnknown pointer, QueryInterface() it to
	# IEnumVARIANT
	from comtypes.automation import IEnumVARIANT
	return enum.QueryInterface(IEnumVARIANT)

These correspond to one class per attribute name that is _NewEnum, Count or Item.
After all, I think it is unlikely that any one of these will be patched, and all three will usually be patched in order to implement COM collection class behavior.

In any case, this implementation is a bad fit with static type analysis.

I'm sure you have some great ideas for this too.

[junkmd]

I will close this issue since it has been shifted to #400 and #401, related to inline type annotations.

The scope of this issue resulted in only adding PEP484-compliant type annotations for statically defined modules and refactoring of the module and code generation process.

@kdschlosser
Please help us with #400 and #401 as well.