Update pydantic to ~=1.8

[CasperWA]

Closes #730.

It seems we do not need to support more of the special OpenAPI keys that we're already supporting. Only allOf is used in the first-level for the retrieving the "attributes" fields anyway.

This PR also closes #729, since it will update pydantic, adding proper support for Python 3.9.

[CasperWA]

Note, this will update the OpenAPI schemas, but this makes sense, as the Enum classes simply get the same treatment and representation as pydantic models have always had.

[codecov]

Codecov Report

Merging #731 (bf54594) into master (c282454) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #731   +/-   ##
=======================================
  Coverage   92.81%   92.81%           
=======================================
  Files          67       67           
  Lines        3520     3520           
=======================================
  Hits         3267     3267           
  Misses        253      253           

Flag	Coverage Δ
project	92.81% <ø> (ø)
validator	63.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
optimade/models/entries.py	97.50% <ø> (ø)
optimade/models/index_metadb.py	100.00% <ø> (ø)
optimade/models/links.py	100.00% <ø> (ø)
optimade/models/structures.py	96.38% <ø> (ø)
optimade/models/utils.py	91.56% <ø> (ø)
...made/server/entry_collections/entry_collections.py	98.85% <ø> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c282454...bf54594. Read the comment docs.

[CasperWA]

Added a test for the get_attribute_fields() method. The test could also inspire a different way of determining the attribute fields in general, if one can get a hold of the relevant resource attribute pydantic model... :)

Otherwise, it's ready for review @ml-evs.

[CasperWA]

It's worth mentioning @rartino and possibly @merkys as well for this PR, as we're slightly changing the OpenAPI schema (which will later be passed onto the central OPTIMADE repository).

What: The change is for schema fields that use enumerations only, adding information about the default value and field descriptions - something that is already present for other non-enumeration schema fields. It also uses the allOf OpenAPI key to reference the enumeration model. This is a discussed topic in the the OpenAPI community, but is definitely a valid representation (relevant documentation).

Why: This is the updated way of handling Python Enum classes when used as the field types in pydantic models (pydantic being the Python package we use to define the schema as Python data classes). So if we want to keep up with the latest version of pydantic and not be left behind, this is more-or-less a mandatory change. In theory it can be changed with a dedicated custom serializer function, but this is a lot of effort to put in.

[ml-evs]

I'm fine with this, but for continuity I think it makes sense to release the 1.0.1 of the specification before we make this change (I have just merged the most recent schema sync PR). Will review the new code soon.

[CasperWA]

I'm fine with this, but for continuity I think it makes sense to release the 1.0.1 of the specification before we make this change (I have just merged the most recent schema sync PR). Will review the new code soon.

Yes and no. It shouldn't take long to update the schemas in the specification repo again immediately if we just hail the right people for 5 min. :) But to find out how long time it would take, we would first need an evaluation of this PR by those right people (hint @rartino, @merkys, ...) 😄 (specifically revieweing the OpenAPI schemas.)

[merkys]

I have reviewed OpenAPI schemas and I am fine with their changes. In Python code I see only a single change in comment and addition of a test, which is almost always a good thing. Thus I tend to approve this PR.

As for updating the OpenAPI schemas for OPTIMADE, I am fine with both pre-1.0.1 and post-1.0.1 merge. Pre-1.0.1 merge is sensible due to specified default value for aggregate.

[CasperWA]

@ml-evs I would still need you approval here as one of the code owners. Also, if you agree I will open another PR to update the schemas (again) in the OPTIMADE repository and we can try and quickly get this update in, including a release to v0.13.3.

[CasperWA]

Ah! I found another "issue". The "title" OpenAPI key is not present for properties using Enum types. Indeed, it is now removed from structure_features and others in the openapi.json file. Note also that it was never present for the links attributes that use Enum types, so this is an opportunity to add them for all properties using Enums.

[CasperWA]

@merkys I have fixed the schemas according to the comment above. You can see the changes with respect to the ones that you accepted here and see the aggregated changes by reviewing normally. Note, that there are no longer changes in any "title" values there, only additions.

[ml-evs]

Thanks @CasperWA, looks good.

[CasperWA]

Thanks @CasperWA, looks good.

I'll merge this in and open a PR in the OPTIMADE repository, discussing the extra changes since @merkys reviewed and accepted this one previously there.