Clarify type buffer protocol type annotations & associated limits

[pushfoo]

Enhancement request:

What should be added/changed?

Einarf added support for arbitrary buffer protocol implementations as arguments to arcade.gl.Texture.write in #703.

Everything works as described when executing. However, PyCharm's linter reports passing a memoryview instance to write as an incorrect type.

There is currently no type hinting support for arbitrary buffer protocol implementations, but there is a promising recent PEP.

In the meantime, we can change write's signature to support a reasonable range of valid buffer protocol objects in PyCharm.

arcade/arcade/gl/texture.py

Line 643 in 225209a

def write(self, data: Union[bytes, Buffer, array], level: int = 0, viewport=None) -> None:

Replacing the above signature with the following seems to make PyCharm's default linter behave better:

# typing.ByteString was marked deprecated in Python 3.9, but this has
# long-term support while also backward-compatible with Python 3.7
from collections.abc import ByteString

class Texture:
    def write(self, data: Union[ByteString, memoryview, Buffer, array], level: int = 0, viewport=None) -> None:

Alternatively, we could add our own Union alias variable and throw everything reasonable in there. We definitely can't use protocols since we have to support 3.7 and they were added in 3.8 : https://peps.python.org/pep-0544/

What would it help with?

tl;dr we'd get PyCharm compatibility, more readable code, and probably future-proofing

The PEP mentioned earlier appears to outline the following reasons for creating an explicit buffer protocol class:

1. bytes might not stay compatible with memoryview and bytearray
2. The compatibility behavior isn't explicitly clear
3. A catch-all class would be easier to read

[pushfoo]

This probably applies to other classes and functions as well. I'll do a more thorough look through the code later.

[einarf]

Yeap. Was struggling with this. Might as well make in Any for not until the typing stuff is resolved?

[pushfoo]

I'd prefer to avoid using Any again. It makes type mismatches harder to debug, especially for ambitious beginners, intermediate users, and anyone attempting to assist them in the help channels.

After reading up on the default linters used by IDEs, I think some of our initial assumptions were wrong. Past versions might not have supported # type: ignore out of the box, but recent versions of both VSCode (through the pyright linter) and PyCharm appear to. I've tested this on PyCharm CE 2022.1.1, and it appears to work.

How does the following plan look?

1. For now, switch to a Union type as described earlier, along with a warning box in the doc very briefly explaining how and when to use # type: ignore
2. Add or expand documentation about types and type checking during Documentation Reorganization #1194
3. Attempt a more proper fix when annotations for buffer protocol instances become plausible, either through Python itself or through the typing_extensions backport module

I'm not sure when or if the backport module will add support for buffer protocol typing. At the moment, it's a pure Python project with no C. However, their README seems encouraging:

New features may be added to typing_extensions as soon as they are specified in a PEP that has been added to the python/peps repository. If the PEP is accepted, the feature will then be added to typing for the next CPython release. No typing PEP has been rejected so far, so we haven't yet figured out how to deal with that possibility.

The PEP for buffer protocol typing is already in the python/peps repo.

[alejcas]

Just a little note, consider creating an alias for this complex types like:

BufferDataType = Union[ByteString, memoryview, Buffer, array]

And then using BufferDataType everywhere

[pushfoo]

That was mentioned in the OP:

Alternatively, we could add our own Union alias variable and throw everything reasonable in there.

Since @einarf mentioned in discord discussion that there were more places than just this module that may be affected, is there an unexpected place to put this? Or is arcade.arcade_types fine?

[einarf]

Since @einarf mentioned in discord discussion that there were more places than just this module that may be affected, is there an unexpected place to put this? Or is arcade.arcade_types fine?

Should be fine