Support pydantic v2 & test on pre-releases?

[effigies]

I just started testing a project using Pydantic using pip install --pre, and I get the following result when attempting to build documentation:

Running Sphinx v5.0.2

Extension error:
Could not import extension sphinxcontrib.autodoc_pydantic (exception: cannot import name 'BaseSettings' from 'pydantic' (/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/pydantic/__init__.py))
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/sphinx/registry.py", line 430, in load_extension
    mod = import_module(extname)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/sphinxcontrib/autodoc_pydantic/__init__.py", line 11, in <module>
    from sphinxcontrib.autodoc_pydantic.directives.autodocumenters import (
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/sphinxcontrib/autodoc_pydantic/directives/autodocumenters.py", line 10, in <module>
    from pydantic import BaseSettings, BaseModel
ImportError: cannot import name 'BaseSettings' from 'pydantic' (/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/pydantic/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/jupyter_book/sphinx.py", line 114, in build_sphinx
    app = Sphinx(
          ^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/sphinx/application.py", line 223, in __init__
    self.setup_extension(extension)
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/sphinx/application.py", line 380, in setup_extension
    self.registry.load_extension(self, extname)
  File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/sphinx/registry.py", line 433, in load_extension
    raise ExtensionError(__('Could not import extension %s') % extname,
sphinx.errors.ExtensionError: Could not import extension sphinxcontrib.autodoc_pydantic (exception: cannot import name 'BaseSettings' from 'pydantic' (/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/pydantic/__init__.py))

It seems there are some incompatibilities with the upcoming 2.0 release.

[mansenfranzen]

Hi @effigies,

thanks for raising the issue. I was actually expecting to see an issue like this to be opened in regard to the upcoming pydantic v2 release.

In general, I have not started to take a closer look at all the incompatibilities with pydantic v2. Currently, I won't be able to do so, unfortunately 😟.

I'm happy to assist with code reviews and guidance whenever possible. Help wanted! 🙏

I expect that it's going to be a serious effort to make autodoc_pydantic work with the new pydantic major release for 2 main reasons:

• the current implementation of autodoc_pydantic uses several private attributes/methods of pydantic to collect mandatory information which were otherwise not accessible
• pydantic v2 changes many public attributes/methods and surely semantic deviations will also occur

However, there are plenty of tests that help to reveal any incompatibility quickly.

As of yet, there is no final migration guide for pydantic v2. Judging the current state of the docs, it might make sense wait at least a little bit longer. For example, BaseSettings is not even available in v2 yet. Hence, you experience the error you've reported.

[mansenfranzen]

There is currently some great progress regarding pydantic v2 support 😃.

A large body of work was already merged via #160 which makes the tests pass for python >= 3.9 and pydantic v2. This development branch can be found here while the new sphinx docs of autodoc_pydantic can be found here which already showcase all auto-docs features based on pydantic v2.

More work is currently tracked via #162. Any help/review is highly welcome.

[mansenfranzen]

You can now test the new autodoc_pydantic version supporting pydantic v2 via pip install autodoc_pydantic==2.0.0. The dev docs already showcase v2 support (see here). Migration guide is available here.

Any feedback is more than appreciated. The current autodoc_pydantic version 2.0.0 on pypi remains yanked for one week until final release.

[mansenfranzen]

Release for v2 is now on pypi 😃.

You can get the latest version via pip install "autodoc_pydantic>=2.0.1,<3.0".