Allow custom list of domains for inventories

[ssbarnea]

Related: mkdocstrings/mkdocstrings#510
Depends-On: mkdocstrings/mkdocstrings#511

[pawamoy]

Thanks a lot! Just a suggestion.

Could you also update the JSON schema by adding this new domain fields after the other fields of import:

python/docs/schema.json

Line 29 in 96e231a

}

?

[ssbarnea]

Something is off with using a list there as I get a stacktrace...

Traceback (most recent call last):
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/mkdocs/__main__.py", line 250, in build_command
    build.build(cfg, dirty=not clean)
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/mkdocs/commands/build.py", line 311, in build
    env = config.plugins.run_event('env', env, config=config, files=files)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/mkdocs/plugins.py", line 520, in run_event
    result = method(item, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/mkdocstrings/plugin.py", line 243, in on_env
    for page, identifier in collections.ChainMap(*(fut.result() for fut in self._inv_futures)).items():
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/mkdocstrings/plugin.py", line 243, in <genexpr>
    for page, identifier in collections.ChainMap(*(fut.result() for fut in self._inv_futures)).items():
                                                   ^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'list'

As soon I am trying to add a list for import in config, it explodes. Passing string works fine...

[pawamoy]

Oh, that's unfortunate 🤔
Maybe we should cast lists into tuples before passing them to pool executor then. But that's a change in mkdocstrings. I'll try it a bit later.

[pawamoy]

OK, the culprit is functools.lru_cache which does not like unhashable arguments: https://github.com/mkdocstrings/mkdocstrings/blob/master/src/mkdocstrings/plugin.py#L303

Casting lists to tuples fixes the issue. Will push it tomorrow 🙂

[ssbarnea]

First let me thank you for helping on this one, especially as I am new to the project. I wonder if the decision to split mkdocstrings and mkdocstrings-python in two projects did pay off because I already spotted a little bit of need to update both. At least for me it made harder to understand how the code was used. I think i need to put both projects into a parent directory to be able to grep on both at once.

[pawamoy]

The split is useful, though maybe not as clean as it should have been. mkdocstrings, autorefs and the python handler are interlinked, and it makes it indeed quite hard to build a mental model of it. But I'm sure we can make it cleaner with a bit of work 🙂

Ideally, mkdocstrings should provide a generic, solid base for handlers, and changes into one wouldn't need changes into the other.

When debugging things like that, I generally go into the mkdocstrings or mkdocstrings-python project, run a mkdocs build debug session in VSCode and play with both their sources in src or __pypackages__ (or .venv if you configured PDM to install in a venv).

[ssbarnea]

@pawamoy Please take a look at mypy error, it seems unrelated to me:

  > mypy --config-file config/mypy.ini src tests duties.py docs
  src/mkdocstrings_handlers/python/handler.py:195: error: Argument 2 of "collect" is incompatible with supertype "BaseCollector"; supertype defines the argument type as "Mapping[str, Any]"  [override]
  src/mkdocstrings_handlers/python/handler.py:195: note: This violates the Liskov substitution principle
  src/mkdocstrings_handlers/python/handler.py:195: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
  src/mkdocstrings_handlers/python/handler.py:238: error: Argument 2 of "render" is incompatible with supertype "BaseRenderer"; supertype defines the argument type as "Mapping[str, Any]"  [override]
  src/mkdocstrings_handlers/python/handler.py:238: note: This violates the Liskov substitution principle
  src/mkdocstrings_handlers/python/handler.py:238: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
  Found 2 errors in 1 file (checked 10 source files)

While the error is reported near my change, is not on my code. Is that a mypy bug or something else?

Because you do not pin the version of mypy used during testing, weird behaviors can happen. Linters should be pinned, or we would face random failures when they release new version. Based on my experience %30 of mypy/pylint/flake8 updates would require one extra change to the codebase, and they release new versions,... often enogh.

[pawamoy]

LGTM, thanks a lot!