__all__ in a stub will easily go out of date

[AlexWaygood]

Stubtest does not check that the elements in a stub file's __all__ match the elements in the runtime's __all__. It's also not obvious (to me, at least) how stubtest could be patched so that it would check that the elements match.

If __all__ is a tuple, stubtest will raise an error if the stub's __all__ is a different length to the runtime's __all__, but it will not raise an equivalent error if __all__ is a list.

flake8 will raise an error if a name in __all__ is not defined in the file. flake8-pyi on master (not yet released) will also raise an error if __all__ is merely annotated as list[str] without being defined. But flake8 will not raise an error if a name is missing from __all__.

All this means that the __all__ definitions in typeshed are liable to become outdated very easily if new elements are added to __all__ in future Python versions. I'm not sure exactly what the solution is.

[AlexWaygood]

Suggested solution here: python/mypy#12214

But it's too noisy at the moment, due to the number of modules that have __all__ at runtime, but not in the stub. So, it might have to wait a while.