If class is aliased out of a module, explicitly listed members not rendered into documentation

[femtomc]

Describe the bug
Explicitly listing members, but their documentation is not being shown (and code which I'm executing using markdown-exec is not being shown).

Expected behavior
Expect the members to show up in the documentation.

Screenshots

System (please complete the following information):

"mkdocstrings-python @ git+https://github.com/pawamoy-insiders/[email protected]"

• Python: 3.11.3
• OS: Reproduced this for my documentation on Ubuntu, as well as MacOS.

Additional context
Here's my mkdocstrings settings from my mkdocs.yml:

  - mkdocstrings:
      enabled: !ENV [ENABLE_MKDOCSTRINGS, true]
      custom_templates: templates
      default_handler: python
      handlers:
        python:
          options:
            show_source: true
            show_root_heading: true
            show_root_full_path: false
            show_symbol_type_heading: true
            show_symbol_type_toc: true

Here's a log when I serve the documentation:

nox > python -m pip install --constraint=.nox/docs-serve/tmp/requirements.txt mkdocs
nox > python -m pip install --constraint=.nox/docs-serve/tmp/requirements.txt 'mkdocs-material @ git+https://github.com/probcomp/mkdocs-material-insiders'
nox > python -m pip install --constraint=.nox/docs-serve/tmp/requirements.txt 'mkdocstrings-python @ git+https://github.com/pawamoy-insiders/[email protected]'
nox > mkdocs serve
INFO    -  DeprecationWarning: warning_filter doesn't do anything since MkDocs 1.2 and will be
           removed soon. All messages on the `mkdocs` logger get counted automatically.
             File
           "/home/femtomc/Research/genjax/.nox/docs-serve/lib/python3.11/site-packages/mkdocstrings/loggers.py",
           line 9, in <module>
               from mkdocs.utils import warning_filter
             File
           "/home/femtomc/Research/genjax/.nox/docs-serve/lib/python3.11/site-packages/mkdocs/utils/__init__.py",
           line 453, in __getattr__
               warnings.warn(
INFO    -  Building documentation...
INFO    -  Cleaning site directory
INFO    -  The following pages exist in the docs directory, but are not included in the "nav"
           configuration:
             - index.md
             - homepage.md
             - genjax/diff_jl.md
             - genjax/language_aperitifs.md
             - genjax/concepts/generative_functions.md
             - genjax/library/diff_prog/index.md
             - genjax/library/diff_prog/adev.md
             - genjax/library/diff_prog/state.md
             - genjax/library/generative_functions/distributions/coryx.md
             - genjax/library/generative_functions/distributions/gensp.md
             - genjax/library/inference/index.md
             - genjax/library/inference/is.md
INFO    -  Doc file 'index.md' contains an absolute link '/genjax/notebooks/index.html', it was
           left as is.
INFO    -  Doc file 'genjax/notebooks.md' contains an absolute link
           '/genjax/notebooks/index.html', it was left as is.
INFO    -  DeprecationWarning: invalid escape sequence '\s'
             File
           "/home/femtomc/Research/genjax/.nox/docs-serve/lib/python3.11/site-packages/griffe/agents/visitor.py",
           line 177, in get_module
               top_node = compile(self.code, mode="exec", filename=str(self.filepath),
           flags=ast.PyCF_ONLY_AST, optimize=1)
             File "/home/femtomc/Research/genjax/src/genjax/_src/core/datatypes/generative.py",
           line 1110, in
               """> Given a `key: PRNGKey` and arguments `x: Tuple`, the generative
INFO    -  DeprecationWarning: invalid escape sequence '\l'
             File
           "/home/femtomc/Research/genjax/.nox/docs-serve/lib/python3.11/site-packages/griffe/agents/visitor.py",
           line 177, in get_module
               top_node = compile(self.code, mode="exec", filename=str(self.filepath),
           flags=ast.PyCF_ONLY_AST, optimize=1)
             File "/home/femtomc/Research/genjax/src/genjax/_src/core/datatypes/generative.py",
           line 1227, in
               """> Given a `key: PRNGKey`, a choice map indicating constraints ($u$),
INFO    -  DeprecationWarning: invalid escape sequence '\l'
             File
           "/home/femtomc/Research/genjax/.nox/docs-serve/lib/python3.11/site-packages/griffe/agents/visitor.py",
           line 177, in get_module
               top_node = compile(self.code, mode="exec", filename=str(self.filepath),
           flags=ast.PyCF_ONLY_AST, optimize=1)
             File "/home/femtomc/Research/genjax/src/genjax/_src/core/datatypes/generative.py",
           line 1266, in
               """> Given a `key: PRNGKey`, a complete choice map indicating
WARNING:jax._src.xla_bridge:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
INFO    -  Documentation built in 3.77 seconds

[femtomc]

I've found the issue -- in my documentation *.md source, I'm referring to an aliased version of these data structures out of another module.

Seemingly, this is causing an issue for resolution for griffe? And perhaps this is already issue'd elsewhere (I just missed this).

[femtomc]

The above strikes me as weird...

Seemingly, griffe gets to the source (here's another example):

So e.g. it resolves the alias to get the source code -- but mkdocstrings is having issues. If I explicitly change the documentation to refer to the "inner" module (e.g. in my project, code which I'm hiding under a _src directory) -- it works again.

[femtomc]

@pawamoy Btw, I swear this might be from a recent commit -- because I was successfully building and showing members from aliased (out of a module) names yesterday morning 😄

[pawamoy]

Hi @femtomc, thanks for the report! I'll investigate with what you gave me and come back to you if I need more info 🙂

[femtomc]

Is this perhaps a recent change? https://mkdocstrings.github.io/griffe/reference/griffe/?h=public#griffe.Attribute.is_public

My best guess as to what I missed.

Edit: testing now.

[pawamoy]

You're right, that highly possible. Just after our email exchange, I released mkdocstrings-python Insiders 1.5.1.1.3.0, you could try with that version, I think it will fix your issue.

The explanation is: Griffe 0.35 returns members of aliases as aliases themselves. Previously the actual, resolved members were returned. This change was needed to preserve the path (the alias path, not the canonical one) of alias members. And an update was needed in mkdocstrings-python to support that.

[femtomc]

Okay will try! One moment, thanks!

[femtomc]

@pawamoy Should I set my inner (original) dataclasses .public attribute to True?

Ah, I see -- so my dataclasses should automatically be public if I export them via __all__. I think I'm doing this correctly, will add more to issue if the above change doesn't fix.

[pawamoy]

@pawamoy Should I set my inner (original) dataclasses .public attribute to True?

If you refer to what is described in Attribute.is_public, quoting:

If the object's public attribute is set (boolean), return its value.

...then no, by "object" here we refer to the Griffe object (Module/Class/Function/Attribute), not the actual object that is represented (your dataclass).

The public attribute (which by the way is undocumented I realize, will fix) is only meant to "force" an object to be considered public, and is only meant to be used by Griffe extensions, for example to support decorators that mark an object as public.

be public if I export them via all

Yes! If you declare __all__, it is used to know if something is public. When it's not declared, Griffe falls back on additional logic. Note that mkdocstrings-python does not yet only use __all__ when it is declared. We plan on adding an option to do just that, i.e. pick up members to render from __all__ directly, without guessing the public ones.

[femtomc]

I think either (a) something might still be wrong or (b) I'm misunderstanding how to document:

Here, in GenerativeFunction -- I actually define the members and documentation which are showing up in JAXGenerativeFunction

In the markdown source:

::: genjax.core.GenerativeFunction
    options:
      members: 
        - simulate
        - propose
        - importance
        - assess
        - update

The interface definitions of generative functions often interact with JAX tracing machinery. GenJAX does not strictly impose this requirement, but does provide a generative function subclass called `JAXGenerativeFunction` which provides default compatibility definitions for JAX tracing.

Other generative function languages which utilize callee generative functions can enforce JAX compatibility by typechecking on `JAXGenerativeFunction`.

::: genjax.core.JAXGenerativeFunction

Both are exported via an __all__ together -- although there is one final toplevel module which from .reference_module import * which exports the full public API.

[pawamoy]

Thanks for the update. Is your repo public? Otherwise I'll run some tests with dummy code 👍

[femtomc]

@pawamoy It is not public, but I could invite you if you want to do tests with it locally.

I have to run now to do some errands, but let me know if that will help! Very easy for me to do.

[pawamoy]

If that poses no issue to you or your peers/company, then sure, a temporary invitation is perfect 🙂

[femtomc]

That's fine! If you want to interact with me about the repo, let's do so via an issue which you start at the repo itself :) thanks!

[pawamoy]

This was a regression in the children.html template. The conditions that checked whether members were specified or the object was public were wrongly written. This is fixed in mkdocstrings-python 1.5.2 and Insiders 1.5.2.1.3.0. Sorry for the trouble, and thanks for the report, and for your patience!