ast can not parse annotations

[dfuchsgruber]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

Typeguard version

4.0.0rc5

Python version

3.10

What happened?

Typeguard does not support annotations of the form "a b c", which is a format used by jaxtyped, another library that depends on typeguard. Instead, ast throws an error as it is unable to parse the annotation.

How can we reproduce the bug?

Run the code snippet:

from typing_extensions import Annotated
from typeguard import typechecked

@typechecked
def foo(x: Annotated[int, 'a b c']):
    return 1

Which throws

    @typechecked
     ^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_decorators.py", line 213, in typechecked
    retval = instrument(target)
             ^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_decorators.py", line 54, in instrument
    instrumentor.visit(module_ast)
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/ast.py", line 418, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 561, in visit_Module
    self.generic_visit(node)
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/ast.py", line 494, in generic_visit
    value = self.visit(value)
            ^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/ast.py", line 418, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 672, in visit_FunctionDef
    annotation = self._convert_annotation(deepcopy(arg.annotation))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 546, in _convert_annotation
    new_annotation = cast(expr, AnnotationTransformer(self).visit(annotation))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 339, in visit
    new_node = super().visit(node)
               ^^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/ast.py", line 418, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 391, in visit_Subscript
    items = [self.visit(item) for item in slice_value.elts]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 391, in <listcomp>
    items = [self.visit(item) for item in slice_value.elts]
             ^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 339, in visit
    new_node = super().visit(node)
               ^^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/ast.py", line 418, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/site-packages/typeguard/_transformer.py", line 442, in visit_Constant
    expression = ast.parse(node.value, mode="eval")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/staff-ssd/fuchsgru/miniconda3/envs/graph_active_learning/lib/python3.11/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 1
    a b c
      ^
SyntaxError: invalid syntax

[agronholm]

Yeah, the AST transformer should not try to parse the second argument of Annotated.

[agronholm]

Fix released in v4.0.0rc6.

[patrick-kidger]

I'm afraid this is still incompatible with third-party libaries (like jaxtyping). E.g.:

from typeguard import typechecked
from jaxtyping import Float, Array

@typechecked
def foo(x: Float[Array, 'a b c']):
    return 1

A type annotation can be pretty much anything, after all.
(Reproduced from master.)

[agronholm]

How is it supposed to tell the difference between forward references and arbitrary arguments then?

[agronholm]

In this particular case I could check for the presence of whitespace, but that's just a stopgap measure.

[agronholm]

Ok, so what about this?

from typeguard import typechecked
from jaxtyping import Float, Array

@typechecked
def foo(x: Float[Array, 'dim1']):
    return 1

def dim1():
	pass

How does a type checker know that 'dim1' is not supposed to be treated as a forward reference?

[patrick-kidger]

Perhaps demand that all forward references be entirely strings, e.g. have "Foo[Bar]" be typecheckable, but Foo["Bar"] be not? That would also work cleanly with PEP 563. (Bad idea though it was, and hopefully something eventually happens with PEP 649 instead.)

Alternatively, all I'm really expecting from a typecheck against a user-provided class is isinstance(x, annotation), so perhaps only try to resolve forward references inside generics etc?

FWIW I share your frustration. This always feels like fighting the type system.

[agronholm]

Perhaps demand that all forward references be entirely strings, e.g. have "Foo[Bar]" be typecheckable, but Foo["Bar"] be not?

I have lots of uses of forward references within my own apps where I use string forward references only for unresolveable (e.g. typeshed) types. I wouldn't want to declare those invalid.

Alternatively, all I'm really expecting from a typecheck against a user-provided class is isinstance(x, annotation), so perhaps only try to resolve forward references inside generics etc?

How would Float[Array, 'dim1'] differ from, say, Union[int, 'dim1] or Tuple[int, 'dim1'] from the type checker's standpoint? These parameters could just as well be forward references for user provided types too. How am I to tell when to resolve them as a forward reference and when not to? Even if I disabled eager resolution in the AST transformation, all it would do is to punt it to run-time, making it a lot slower for slam-dunk cases (which the vast majority of cases are).

[patrick-kidger]

def resolve_forward_references(annotation):
    if Generic in type(annotation).__mro__ or Protocol in type(annotation).__bases__:
        return True
    origin = typing.get_origin(annotation)
    if origin is not Annotated:
        origin_module = getattr(origin, "__module__", None)
        if origin_module in {"typing", "typing_extensions", "types"}:
            return True
    return False

perhaps? (Untested)

I think that should handle all of Tuple and Union and X | Y and Generic and Protocol and Annotated.

[patrick-kidger]

Actually I think you need a bit more to make Generic and Protocol work, but you get the spirit of it.

[agronholm]

Are you suggesting that typeguard should never eagerly resolve forward references then? That would catastrophically slow down many type checking use cases, as the resolution would have to be done again on every check.

[patrick-kidger]

No. My code snippet above is meant to illustrate when a forward reference should be eagerly resolved.

If it's part of a Python built-in typing object, then it should be resolved.

If it's part of some other annotation (which can be arbitrary), then it shouldn't be resolved.

[agronholm]

The AST transformer can only see names; those names might not have been defined at that point yet, and might never be (due to guarding by if TYPE_CHECKING:. If typeguard were to import them, that would essentially undo the benefits of from __future__ import annotations and cause problems with circular imports, which the aforementioned mechanism is used to work around. My point is, having typeguard try to import modules where these names are potentially defined is hazardous at best, catastrophic at worst, and may not work at all (think of _typeshed imports).

[patrick-kidger]

Run the snippet I wrote above at decoration time, prior to whatever AST magic you're invoking.

I'm not suggesting trying to import any extra modules.

I think trying to resolve anything inside TYPE_CHECKING would be a mistake, to my mind that flag is for static type checking only.

[dkamm]

Hey guys- this might be a dumb idea, but maybe typeguard could allow users to whitelist certain types from forward reference resolution?

Maybe something like this? not sure if there's a better way

if isinstance(slice_value, Tuple):
    if self._memo.name_matches(node.value, *annotated_names):
        # Only treat the first argument to typing.Annotated as a potential
        # forward reference
        items = [self.visit(slice_value.elts[0])] + slice_value.elts[1:]
    elif self._memo.name_matches(node.value, *self.transformer.skip_forward_ref_resolution_names):
        items = list(slice_value.elts)
    else:
        items = [self.visit(item) for item in slice_value.elts]

[agronholm]

Well, Annotated is already hard coded to be on such a whitelist, but as for the jaxtyping names, typeguard would need to statically analyze the source to determine that it's derived from Annotated, since at run time it's something completely different. I don't like doing extra work for a workaround; I'd rather put in more work for a real solution.

[dkamm]

Ah I see- a static analysis solution sounds like the way to go.

FWIW, the jaxtyping types aren't derived from Annotated (and I don't think you can derive from it actually?). They have their own metaclass that makes them subscriptable, which is why I think Patrick suggested distinguishing between typing (and everything that derives from it) vs everything else.

Perhaps there is a way to implement the policy that Patrick suggested, except with static analysis?

It sounds like there's still a question of how you'd want to handle the non-typing types. These are the options I can think of:

1. Do not forward reference resolve the non-typing types
2. Have a config flag for whether to forward reference resolve the non-typing types
3. Allow user to whitelist certain names from forward reference resolving in config
4. Come up with some way to make the distinction during static analysis that authors of libraries like jaxtyping can implement

It seems like the following case could break options 1 and 2, but I'm not sure if something like this would ever come up

from typing import Generic, TypeVar
import torch

K = TypeVar('K')
V = TypeVar('V')
class MyMap(Generic[K, V]):
    pass

class MyType(type):
    pass

class MyClass(metaclass=MyType):
    pass
    
class Foo:
    pass

# annotation is derived from typing, should forward reference resolve
def foo(x: MyMap[str, "Foo"]):
    pass

# annotation is not derived from typing, should not forward reference resolve
def bar(x: jaxtyping.Float[torch.Tensor, "a b c"]):
    pass

# annotation is not derived from typing, should forward reference resolve
def baz(x: MyClass["Foo", int, str]):
    pass

Option 3 would solve the above but is an awkward workaround. I'm not sure how option 4 would work.

[patrick-kidger]

FWIW I'm most in favour of option 1.
(And indeed something needs to be done about generics as in your post.)

Right now, one of the nice things about jaxtyping is that it's compatible out-of-the-box with essentially any runtime type checker (including typeguard versions strictly less than 3). It just provides the types with the appropriate __instancecheck__ methods.

It would be better not to need conditional imports to try and register jaxtyping with each of N different runtime type-checking libraries.

[agronholm]

FWIW, the jaxtyping types aren't derived from Annotated (and I don't think you can derive from it actually?).

Yes, they are, at least in the context of static type checking: https://github.com/google/jaxtyping/blob/main/jaxtyping/indirection.py

FWIW I'm most in favour of option 1.
(And indeed something needs to be done about generics as in your post.)

Options 1 and 3 don't seem very appealing. Option 2, if made opt-in, would at least be easy to implement while still being a crutch. Option 4 is probably the "right" choice (but requires the most work). Jaxtyping already has such code in place (see above).

[patrick-kidger]

Yes, they are, at least in the context of static type checking

Right, but we're talking about runtime type checking here. Annotated is not used at runtime.

Jaxtyping already has such code in place (see above).

?

---

Anyway, I get the sense that this feature - compatibility with arbitrary user annotations -- is not something you're that interested in. And that's fine, of course, we're all just doing this in our spare time.

For now I'll update the jaxtyping docs to recommend beartype only, and we can revisit this if things ever change.

[agronholm]

Right, but we're talking about runtime type checking here. Annotated is not used at runtime.

Option 4 was about having typeguard statically analyze the modules to obtain this information.

[agronholm]

Anyway, I get the sense that this feature - compatibility with arbitrary user annotations -- is not something you're that interested in. And that's fine, of course, we're all just doing this in our spare time.

I don't know where you got that from. It's just that I've done a TON of work to get typeguard to even this point, 6 release candidates...and only THEN do I get told that my approach didn't work, and requires a huge amount of extra work. It's a bit mentally exhausting.

[patrick-kidger]

Option 4 was about having typeguard statically analyze the modules to obtain this information.

Ah, right! FWIW I think this would defeat much of the advantage of runtime type-checking, in that it allows you to communicate things beyond that which the static type system can express. (In jaxtyping's case, shape and dtype information.)

It's a bit mentally exhausting.

I sympathize!

[agronholm]

Ah, right! FWIW I think this would defeat much of the advantage of runtime type-checking, in that it allows you to communicate things beyond that which the static type system can express. (In jaxtyping's case, shape and dtype information.)

Maybe I'm not following. Static analysis would only be used to obtain information on the types that are to be used in type checks.

[patrick-kidger]

I pretty frequently use TYPE_CHECKING to dummy out types that go beyond what static type checking can handle, e.g.

if TYPE_CHECKING:
    Foo: TypeAlias = Any
else:
    class MetaFoo(type):
        def __instancecheck__(cls, obj):
            ...
    class Foo(metaclass=MetaFoo):
        pass


def f(x: Foo):
    ...

I.e. I believe runtime type checking should not respect the TYPE_CHECKING flag.

Indeed to my mind this is the whole purpose of runtime type checking: to add additional checks for the things my static type checker (which I use as well) isn't expressive enough to catch.

[agronholm]

I.e. I believe runtime type checking should not respect the TYPE_CHECKING flag.

That would shut down option 4. What do you think would be the ideal long term solution then?

[patrick-kidger]

So my understanding is that typeguard v4 performs some voodoo magic, parsing AST and modifying it to insert the appropriate type checks (I assume for runtime speed reasons). Meanwhile that typeguard v2 used to (a) perform type checks at the last possible moment, without any processing at decoration time, and (b) used the actual Python objects, not AST.

At least on that basis, I think the ideal solution would loosely be something like the following code:

def typechecked_v5(fn):
    try:
        typechecked_fn = typechecked_v4(fn)
    except Exception:
        @ft.wraps(fn)
        def typechecked_fn(*args, **kwargs):
            return typechecked_v2(fn)(args, **kwargs)
    return typechecked_fn

That is: do whatever voodoo is possible to speed things up where possible, but if necessary fall back to the slow-but-correct thing.

• In practice the try/except is ugly and perhaps this is better tackled by iterating over the annotations at decoration time, and figuring out if they're on a whitelisted happy path, but I'm just trying to communicate the broad strokes here.
• Likewise I'm not suggesting maintaining two whole separate typechecking approaches; I'm guessing that much of the underlying checking machinery is probably be shared between the v4 and v2 style approaches.
• Besides handling jaxtyping-style annotations, I think this might also help with forward references that don't yet exist at decoration time. (I've not checked typeguard's behaviour here too closely -- I know beartype struggles with this one.)

[dkamm]

Hey Patrick, I think this example might be illustrative. (Alex feel free to correct me if I'm wrong)

Here is how typeguard currently transforms the code:

# ==================Original===================
class FooMeta(type):
    def __getitem__(self, x: str) -> None:
        return x

class Foo(metaclass=FooMeta):
    pass

def foo(x: Foo[str, 'Foo']) -> None:
    pass

# ==================Generated==================
def foo(x: Foo[str, 'Foo']) -> None:
    from typeguard import TypeCheckMemo
    from typeguard._functions import check_argument_types
    memo = TypeCheckMemo(globals(), locals())
    check_argument_types('foo', {'x': (x, Foo[str, Foo])}, memo) # Notice 'Foo' has been replaced with Foo

Note that if we instead used Foo[str, 'foo bar'] as the annotation, the code transformation fails because it will try to write Foo[str, foo bar] which is not valid python.

I think solving issue is a matter of getting typeguard to not rewrite Foo[str, 'Foo'] -> Foo[str, Foo] in the check_argument_types call. We need to figure out how to do this during the ast transformation, because we still want to remove the string for the typing cases (like rewriting Union[str, 'Foo'] -> Union[str, Foo]).

For brevity, here's the desired behavior as I understand (with Union and jaxtyping)

# ==================Original===================
class Foo:
    pass

def foo(x: Union[str, 'Foo']) -> None:
    pass

# ==================Generated==================
def foo(x: Union[str, 'Foo']) -> None:
    from typeguard import TypeCheckMemo
    from typeguard._functions import check_argument_types
    memo = TypeCheckMemo(globals(), locals())
    check_argument_types('foo', {'x': (x, Union[str, Foo]])}, memo) # 'Foo' has been rewritten to Foo

# ==================Original===================
import torch
from jaxtyping import Float

def foo(x: Float[torch.Tensor, 'batch']) -> None:
    pass

# ==================Generated==================
def foo(x: Float[torch.Tensor, 'batch']) -> None:
    from typeguard import TypeCheckMemo
    from typeguard._functions import check_argument_types
    memo = TypeCheckMemo(globals(), locals())
    check_argument_types('foo', {'x': (x, Float[torch.Tensor, 'batch'])}, memo) # 'batch' has not been rewritten

I think for option 4, if there were some way to inspect a subscriptable annotation in the ast transformer to determine whether to rewrite its string elements or not, that would work. Maybe it could be some special flag on the AbstractDType class that the ast transformer could inspect

[dkamm]

BTW, I'm down to contribute to both typeguard and jaxtyping to make this happen 🙂 (assuming we want to go that way and you guys are cool with it)

[agronholm]

BTW, I'm down to contribute to both typeguard and jaxtyping to make this happen slightly_smiling_face (assuming we want to go that way and you guys are cool with it)

I would very much like to get some kind of a solution. If you want to contribute, I think you should come up with a concrete plan of action first that I could agree on, to avoid any wasted effort.

[patrick-kidger]

@patrick-kidger I think it's up to the typechecker to determine how to process the annotation and we could take advantage of that here? See https://peps.python.org/pep-0593/#consuming-annotations

For instance, it seems quite reasonable for the typechecker to do an isinstance check on the obj with the first element, then if any of the other elements are callable, call them on the obj. Each callable can return None if its check passes and an error string if its check fails, which the typechecker can assemble and return.

This would require changing how typeguard handles Annotated though, and deciding on an interface for the callables. But I don't see why we wouldn't be able to move the __instancecheck__ logic into callables if such changes are made.

This is establishing a standard that we'd be expecting/asking other runtime typecheckers to follow. This isn't actually necessary, as discussed above.

[agronholm]

All options that don't involve adding extra stack frames are on the table.

[dkamm]

@patrick-kidger that's true. I guess my main argument for doing this is that it's a very natural standard that seems to be implied by the spec. Like I wouldn't be surprised if the typecheckers converged on it. Also afaik, the two main runtime typecheckers are typeguard and beartyping and seems like we could coordinate at this stage.

@agronholm the rough plan would be the following

1. Change jaxtyping to be Annotated aliases as described above. This would get rid of the Annotated as Float hack
2. Change typeguard to handle annotations as described above (or by whatever standard we decide on)
3. Detect Annotated aliases using static analysis so we can treat them as Annotated in ast transform (your current plan). I think this would primarily involve assignments but could be quite a few edge cases here.

[dkamm]

Hmm might need to think through this more though. The annotation could contain a callable that returns something else and would need to handle that

[dkamm]

Yeah, this is not a good idea actually. There's a lot of flexibility in how you can use Annotated, and there's no way to distinguish from our perspective. For instance, that struct example in the spec uses the annotation to determine how to unpack the fields.

So I think we're back to the original plan (part 3 of the above). Disregard parts 1 and 2. No changes needed on the jaxtyping side.

In fact, this fix is about really about handling Annotated aliases properly.

Edit: I moved something extra I wrote here to a separate issue in jaxtyping because it's unrelated to typeguard.

[dkamm]

Hey @agronholm just wanted to check if you're already working on this. I did an experiment here https://github.com/dkamm/annotated-alias-detector. Let me know if it's along the lines of what you were thinking

[agronholm]

I'm busy with other projects currently. I'll resume work on typeguard in the coming weeks and get a patch release out. If feasible, I'll do what I can about this.

[Conchylicultor]

Could this bug be re-open ?

Our project is broken by recent version of typeguard.

[agronholm]

What exactly broke it? Also, there was no consensus on the solution. This would be a major undertaking too. Are you willing to contribute code?

[Conchylicultor]

Basically all projects dependent on jaxtyping cannot use the last typeguard version: https://github.com/google/jaxtyping/network/dependents

One easy fix would be to add some __typeguard__ protocol that downstream libraries could use to propagate the annotation to use:

from typeguard import typechecked
from jaxtyping import jaxtyped, Float
from torch import Tensor


x = Float[Tensor, 'a b c']
assert x.__typeguard__()  == Tensor


@jaxtyped
@typechecked
def foo(x: Float[Tensor, 'a b c']):  # typeguard uses `Tensor` annotation
    return 1

[abravalheri]

I pretty frequently use TYPE_CHECKING to dummy out types that go beyond what static type checking can handle, e.g.

if TYPE_CHECKING:
    Foo: TypeAlias = Any
else:
    class MetaFoo(type):
        def __instancecheck__(cls, obj):
            ...
    class Foo(metaclass=MetaFoo):
        pass


def f(x: Foo):
    ...

I.e. I believe runtime type checking should not respect the TYPE_CHECKING flag.

Indeed to my mind this is the whole purpose of runtime type checking: to add additional checks for the things my static type checker (which I use as well) isn't expressive enough to catch.

I usually also rely on TYPE_CHECKING to convince static type checkers (since they cannot handle dynamic hooks like __instancecheck__). I don't think it is possible to use static type checkers in this circumstance without TYPE_CHECKING (as far as I know there is no other alternative).

What I ended up doing as a workaround for typeguard is the following (seems to be working):

_STATIC_TYPE_CHECKING = TYPE_CHECKING  # workaround for typeguard

if _STATIC_TYPE_CHECKING:
    Foo: TypeAlias = Any
else:
    class MetaFoo(type):
        def __instancecheck__(cls, obj):
            ...
    class Foo(metaclass=MetaFoo):  # type: ignore[no-redef]
        pass

@agronholm, is this a reasonable/reliable workaround for the limitation discussed that @patrick-kidger mentioned in
#353 (comment)?

[agronholm]

@agronholm, is this a reasonable/reliable workaround for the limitation discussed that @patrick-kidger mentioned in #353 (comment)?

I don't quite understand how this would solve the problem. How would Typeguard know, at import time, if it should eagerly resolve the parameters of the annotation, or not?

[agronholm]

Also, was this meant as a workaround in jax-typing, or in apps/libraries that use Typeguard?

[abravalheri]

Also, was this meant as a workaround in jax-typing, or in apps/libraries that use Typeguard?

Hi @agronholm, this is a workaround for other apps/libraries that use typeguard and relates specifically to a previous comment in this thread: #353 (comment). Maybe it would also be relevant to jax-typing, but I don't know.

I don't quite understand how this would solve the problem. How would Typeguard know, at import time, if it should eagerly resolve the parameters of the annotation, or not?

Specifically, I am forced to use TYPE_CHECKING to make mypy/other static analysis tools happy¹ because they are simply not able to handle dynamic aspects of the Python programming language.

I was surprised when I noticed typeguard behaved as if the runtime value of TYPE_CHECKING was True², and I found this issue when looking for more information (specifically the comment in #353 (comment)).
For example³:

> docker run --rm -it python:3.9-bullseye /bin/bash
cd /tmp
python3 -m venv .venv
.venv/bin/python -m pip install 'typeguard==4.1.0'

cat <<EOF > script.py
from typing import TYPE_CHECKING, Annotated
from typeguard import typechecked

if TYPE_CHECKING:
    NonEmptyStr = Annotated[str, "Non empty string"]
else:
    class _NonEmptyStrMeta(type):
        def __instancecheck__(cls, obj):
            return isinstance(obj, str) and obj

    class NonEmptyStr(metaclass=_NonEmptyStrMeta):   # type: ignore[no-redef]
        """Non empty string"""

@typechecked
def my_print(msg: NonEmptyStr):
    print("my string:", repr(msg))

print(f"{isinstance('', NonEmptyStr)=}")
my_print("")
EOF

.venv/bin/python script.py

Will output:

isinstance('', NonEmptyStr)=False
my string: ''

This comes as a surprise to me... Since isinstance returns False, I was expecting the call to my_print to raise an exception.

Please note that if I assign the value of TYPE_CHECKING to another variable, I can obtain the expected result:

sed -i 's~if TYPE_CHECKING~_STATIC_TYPE_CHECKING = TYPE_CHECKING\nif _STATIC_TYPE_CHECKING~'  script.py  # GNU sed (newline)
.venv/bin/python script.py
# ...
# typeguard.TypeCheckError: argument "msg" (str) is not an instance of __main__.NonEmptyStr

Note that I can run mypy script.py with the file written this way and not get type errors.
However if I remove the if statement, I can get a (misleading) type error from mypy⁴:

cat <<EOF > script2.py
from typing import TYPE_CHECKING, Annotated
from typeguard import typechecked

class _NonEmptyStrMeta(type):
    def __instancecheck__(cls, obj):
        return isinstance(obj, str) and obj

class NonEmptyStr(metaclass=_NonEmptyStrMeta):
    """Non empty string"""

@typechecked
def my_print(msg: NonEmptyStr):
    print("my string:", repr(msg))

print(f"{isinstance('hello', NonEmptyStr)=}")
my_print("hello")
EOF

.venv/bin/python -m pip install 'mypy==1.4.1'
.venv/bin/mypy script2.py
# script2.py:16: error: Argument 1 to "my_print" has incompatible type "str"; expected "NonEmptyStr"  [arg-type]
# Found 1 error in 1 file (checked 1 source file)

My previous question (is this a reasonable/reliable workaround...?) targets this situation in particular. Is it reasonable/reliable to use a assignment like _STATIC_TYPE_CHECKING = TYPE_CHECKING and then use this new variable in a conditional (if _STATIC_TYPE_CHECKING:) to avoid typeguard from misinterpreting the runtime value of TYPE_CHECKING, as shown in the examples above?

Footnotes

1. Not only for the benefit of my own workflow, but also for other contributors and consumers of the libraries I write. ↩

2. The docs explicitly say "A special constant that is assumed to be True by 3rd party static type checkers. It is False at runtime." (emphasis are mine). In my mind typecheck is a runtime tool, not a static type checker. ↩

3. This is a toy example, merely illustrative. The classes and validation checks are not the same as the ones used in real life. I just wanted something simple to illustrate. ↩

4. I wish mypy would simply understand __instancecheck__ but it does not... It is a fair limitation that static type checks have. typeguard can do better than that, since it can work with runtime information... ↩

[agronholm]

Typeguard statically evaluates the contents of if TYPE_CHECKING: blocks because this is a trick often used to work around circular import problems. Not doing so would cause a lot of breakage. What I might consider is also evaluating the else: blocks. But even this only applies to modules directly inspected by Typeguard. For any other code, only the run-time part is ever seen by Typeguard. There are potential, nontrivial solutions to this, however.

[agronholm]

I am working on some PoC code that tries to resolve the names to determine if they are safe to unquote or not.