Add support levels to validator config

[ml-evs]

This PR adds the concept of support queryability from the model schemas into the validator config.

This allows for some changes in the validator to enable running query checks on specification fields where implementations have not explicitly provided the type in their /info endpoints.

One consequence of this that we can discuss... if a field is present in the /info/<entry> of an implementation, the validator will report a failure in two cases (where it didn't before):

1. If the field has a null value in the structure that is "chosen" to generate queries.
2. If the field has queryable level of "MUST" (even if the field is optional) then this will now be a failure.

I think I'm fine with 2. but 1. feels like more of a grey area...

Still to-do (in this PR or elsewhere):

• Refactor entry schemas import so that server config is not touched during validation (done in Move entry schemas to separate submodule #511)

[codecov]

Codecov Report

Merging #503 into master will decrease coverage by 0.04%.
The diff coverage is 81.81%.

@@            Coverage Diff             @@
##           master     #503      +/-   ##
==========================================
- Coverage   91.59%   91.55%   -0.05%     
==========================================
  Files          62       62              
  Lines        3119     3127       +8     
==========================================
+ Hits         2857     2863       +6     
- Misses        262      264       +2     

Flag	Coverage Δ
#project	91.55% <81.81%> (-0.05%)	⬇️
#validator	63.76% <81.81%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
optimade/validator/validator.py	82.46% <76.47%> (-0.22%)	⬇️
optimade/models/utils.py	90.41% <100.00%> (ø)
optimade/validator/config.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 995f32d...eabb60b. Read the comment docs.

[ml-evs]

This PR now contains the changes from #511, happy to review as one PR or two.

[CasperWA]

Great work @ml-evs

Some suggested changes inbound.

Concerning your OP null or None is considered a valid value for any property by the OPTIMADE API specification, I believe (then of course for some properties it is not allowed, but that always stated clearly). So this should definitely be allowed. But we can of course remark on it, as information, and later remove the remark if developers find it off-putting.

[ml-evs]

Thanks for the review @CasperWA!

Concerning your OP null or None is considered a valid value for any property by the OPTIMADE API specification, I believe (then of course for some properties it is not allowed, but that always stated clearly). So this should definitely be allowed. But we can of course remark on it, as information, and later remove the remark if developers find it off-putting.

I've just addressed this in the last commit: this now only fails if the field is a "MUST" field, otherwise it just skips. This will still fail if the field isn't returned by the implementation by default, but I'm fixing that in #515 as its a bit more involved.

[ml-evs]

Concerning your OP null or None is considered a valid value for any property by the OPTIMADE API specification, I believe (then of course for some properties it is not allowed, but that always stated clearly). So this should definitely be allowed. But we can of course remark on it, as information, and later remove the remark if developers find it off-putting.

I've just addressed this in the last commit: this now only fails if the field is a "MUST" field, otherwise it just skips. This will still fail if the field isn't returned by the implementation by default, but I'm fixing that in #515 as its a bit more involved.

Actually this opens a can of worms (our server would fail it - #516) and needs #504 to be in first... I'll make a separate PR that add this commit to the validator (#517)

[CasperWA]

Looking good, minor things that you can take or leave.

[CasperWA]

Get in