Refactor shallow schema #1350

delapuente · 2018-11-28T00:22:43Z

This PR introduces a kind of mixin field ModelValidation with model type validation. All the Marshmallow fields are imported into qiskit.validation.fields and extended to include the mixin. The same happens with polymorphic fields and containers.

Validation is possible thanks to a special copy of the model schema, the shallow schema, which replaces the _deserialize private method of its fields (at instance-level) to use the new validate_model from the ModelValidator mixin.

Validating models consist into checking that the values are of the corresponding types. Namely:

For regular fields, the type they deserialize to (deserialization type).
For fields containers (such as lists), the deserialization type of the contained field.
For schema containers (such as nested schemas), the type of the schema's model.
For polymorphic fields, one of the deserialization types of the choices.
For polymorphic schemas, one of the types of the possible schemas' model.

While validating, nothing more than the type of the value is considered. Validation assumes that the values of the inner properties are already valid or they would have failed during their instantiation. For instance, consider:

Book(title="The Terror", author=Person(name="Dan Simmons"))

Person instantiation is executed first. If successful, validating Book does not require author internal structure (i.e. name) to be validated again since otherwise, construction of Person would have failed.

Overview ======== This PR introduces a kind of _mixin field_ `ModelValidation` with model type validation. All the Marshmallow fields are imported into `qiskit.validation.fields` and extended to include the mixin. The same happens with polymorphic fields and containers. Validation is possible thanks to a special copy of the model schema, the _shallow schema_, which replaces the `_deserialize` private method of its fields (at instance-level) to use the new `validate_model` from the `ModelValidator` mixin. Validating models consist into checking that the values are of the corresponding types. Specifically: 1. For regular fields, the type they deserialize to (deserialization type). 2. For fields containers (such as lists), the deserialization type of the contained field. 3. For schema containers (such as nested schemas), the type of the schema's model. 4. For polymorphic fields, one of the deserialization types of the choices. 5. For polymorphic schemas, one of the types of the possible schemas' model. While validating, nothing more than the type of the value is considered. Validation assumes that the values of the inner properties are already valid or they would have failed during their instantiation. For instance, consider: ```python Book(title="The Terror", author=Person(name="Dan Simmons")) ``` `Person` instantiation is executed first. If successful, validating `Book` does not require `author`'s internal structure (i.e. `name`) to be validated again since otherwise, construction of `Person` would have failed. --- This refactor continues patching the method `_deserialize` of the schema fields but, instead of deciding how to patch it according to the type, it makes `_deserialize` to call a new custom method `_validate_model` with the desired behaviour. The different strategies from the if/elif/else block of the original implementation are now spread over the fields defined in qiskit.validation.fields This package also contains versions of marshmallow.fields that include their own `_validate_model` method.

This PR split the fields into general fields, containers and polymorphic. Each module has the definition of the different kind of fields. It also provides a base field for those able of model type validation and a general mechanism to perform this validation.

…ons.

…allow-schema

* `shallow_schema` refactor to make its generation more maintainable. Overview ======== This PR introduces a kind of _mixin field_ `ModelValidation` with model type validation. All the Marshmallow fields are imported into `qiskit.validation.fields` and extended to include the mixin. The same happens with polymorphic fields and containers. Validation is possible thanks to a special copy of the model schema, the _shallow schema_, which replaces the `_deserialize` private method of its fields (at instance-level) to use the new `validate_model` from the `ModelValidator` mixin. Validating models consist into checking that the values are of the corresponding types. Specifically: 1. For regular fields, the type they deserialize to (deserialization type). 2. For fields containers (such as lists), the deserialization type of the contained field. 3. For schema containers (such as nested schemas), the type of the schema's model. 4. For polymorphic fields, one of the deserialization types of the choices. 5. For polymorphic schemas, one of the types of the possible schemas' model. While validating, nothing more than the type of the value is considered. Validation assumes that the values of the inner properties are already valid or they would have failed during their instantiation. For instance, consider: ```python Book(title="The Terror", author=Person(name="Dan Simmons")) ``` `Person` instantiation is executed first. If successful, validating `Book` does not require `author`'s internal structure (i.e. `name`) to be validated again since otherwise, construction of `Person` would have failed. --- This refactor continues patching the method `_deserialize` of the schema fields but, instead of deciding how to patch it according to the type, it makes `_deserialize` to call a new custom method `_validate_model` with the desired behaviour. The different strategies from the if/elif/else block of the original implementation are now spread over the fields defined in qiskit.validation.fields This package also contains versions of marshmallow.fields that include their own `_validate_model` method. * Organize the fields into several modules and factor out validation. This PR split the fields into general fields, containers and polymorphic. Each module has the definition of the different kind of fields. It also provides a base field for those able of model type validation and a general mechanism to perform this validation. * Polishing docstrings and reviewing container and polymorphic validations. * Rename to make intentions clear and fixed fields hierarchy * Moving tests to its own folder * Split tests into different categories * Docstrings fixes and clarifications. * Style, convention and doc tweaks * Update CHANGELOG

* moving things * Removing * Moving around * Test * Changeling * Refactor shallow schema (#1350) * `shallow_schema` refactor to make its generation more maintainable. Overview ======== This PR introduces a kind of _mixin field_ `ModelValidation` with model type validation. All the Marshmallow fields are imported into `qiskit.validation.fields` and extended to include the mixin. The same happens with polymorphic fields and containers. Validation is possible thanks to a special copy of the model schema, the _shallow schema_, which replaces the `_deserialize` private method of its fields (at instance-level) to use the new `validate_model` from the `ModelValidator` mixin. Validating models consist into checking that the values are of the corresponding types. Specifically: 1. For regular fields, the type they deserialize to (deserialization type). 2. For fields containers (such as lists), the deserialization type of the contained field. 3. For schema containers (such as nested schemas), the type of the schema's model. 4. For polymorphic fields, one of the deserialization types of the choices. 5. For polymorphic schemas, one of the types of the possible schemas' model. While validating, nothing more than the type of the value is considered. Validation assumes that the values of the inner properties are already valid or they would have failed during their instantiation. For instance, consider: ```python Book(title="The Terror", author=Person(name="Dan Simmons")) ``` `Person` instantiation is executed first. If successful, validating `Book` does not require `author`'s internal structure (i.e. `name`) to be validated again since otherwise, construction of `Person` would have failed. --- This refactor continues patching the method `_deserialize` of the schema fields but, instead of deciding how to patch it according to the type, it makes `_deserialize` to call a new custom method `_validate_model` with the desired behaviour. The different strategies from the if/elif/else block of the original implementation are now spread over the fields defined in qiskit.validation.fields This package also contains versions of marshmallow.fields that include their own `_validate_model` method. * Organize the fields into several modules and factor out validation. This PR split the fields into general fields, containers and polymorphic. Each module has the definition of the different kind of fields. It also provides a base field for those able of model type validation and a general mechanism to perform this validation. * Polishing docstrings and reviewing container and polymorphic validations. * Rename to make intentions clear and fixed fields hierarchy * Moving tests to its own folder * Split tests into different categories * Docstrings fixes and clarifications. * Style, convention and doc tweaks * Update CHANGELOG * Make to_dict and from_dict more discoverable, adjust converter tests (#1428) * Move to_dict and from_dict to BaseModel Move the `to_dict()` and `from_dict()` methods directly to the `BaseModel` instead of appending them at decoration time, for ease of usage and aiding discovery. * Fix converter tests * Result postprocess (#1404) * changelog * add result.postprocessing functions * format_statevector * restore ibmq _reorder_bits * update get_statevector, get_unitary, get_counts * remove the temporary hex conversion of tests * update remaining tests * Use integers in properties.gates.qubits (#1427) Use `integer` instead of `number` in the schema for BackendProperties (`BackendProperties.gates[x].qubits`). * Cleaning up * Linting

* `shallow_schema` refactor to make its generation more maintainable. Overview ======== This PR introduces a kind of _mixin field_ `ModelValidation` with model type validation. All the Marshmallow fields are imported into `qiskit.validation.fields` and extended to include the mixin. The same happens with polymorphic fields and containers. Validation is possible thanks to a special copy of the model schema, the _shallow schema_, which replaces the `_deserialize` private method of its fields (at instance-level) to use the new `validate_model` from the `ModelValidator` mixin. Validating models consist into checking that the values are of the corresponding types. Specifically: 1. For regular fields, the type they deserialize to (deserialization type). 2. For fields containers (such as lists), the deserialization type of the contained field. 3. For schema containers (such as nested schemas), the type of the schema's model. 4. For polymorphic fields, one of the deserialization types of the choices. 5. For polymorphic schemas, one of the types of the possible schemas' model. While validating, nothing more than the type of the value is considered. Validation assumes that the values of the inner properties are already valid or they would have failed during their instantiation. For instance, consider: ```python Book(title="The Terror", author=Person(name="Dan Simmons")) ``` `Person` instantiation is executed first. If successful, validating `Book` does not require `author`'s internal structure (i.e. `name`) to be validated again since otherwise, construction of `Person` would have failed. --- This refactor continues patching the method `_deserialize` of the schema fields but, instead of deciding how to patch it according to the type, it makes `_deserialize` to call a new custom method `_validate_model` with the desired behaviour. The different strategies from the if/elif/else block of the original implementation are now spread over the fields defined in qiskit.validation.fields This package also contains versions of marshmallow.fields that include their own `_validate_model` method. * Organize the fields into several modules and factor out validation. This PR split the fields into general fields, containers and polymorphic. Each module has the definition of the different kind of fields. It also provides a base field for those able of model type validation and a general mechanism to perform this validation. * Polishing docstrings and reviewing container and polymorphic validations. * Rename to make intentions clear and fixed fields hierarchy * Moving tests to its own folder * Split tests into different categories * Docstrings fixes and clarifications. * Style, convention and doc tweaks * Update CHANGELOG

* moving things * Removing * Moving around * Test * Changeling * Refactor shallow schema (Qiskit#1350) * `shallow_schema` refactor to make its generation more maintainable. Overview ======== This PR introduces a kind of _mixin field_ `ModelValidation` with model type validation. All the Marshmallow fields are imported into `qiskit.validation.fields` and extended to include the mixin. The same happens with polymorphic fields and containers. Validation is possible thanks to a special copy of the model schema, the _shallow schema_, which replaces the `_deserialize` private method of its fields (at instance-level) to use the new `validate_model` from the `ModelValidator` mixin. Validating models consist into checking that the values are of the corresponding types. Specifically: 1. For regular fields, the type they deserialize to (deserialization type). 2. For fields containers (such as lists), the deserialization type of the contained field. 3. For schema containers (such as nested schemas), the type of the schema's model. 4. For polymorphic fields, one of the deserialization types of the choices. 5. For polymorphic schemas, one of the types of the possible schemas' model. While validating, nothing more than the type of the value is considered. Validation assumes that the values of the inner properties are already valid or they would have failed during their instantiation. For instance, consider: ```python Book(title="The Terror", author=Person(name="Dan Simmons")) ``` `Person` instantiation is executed first. If successful, validating `Book` does not require `author`'s internal structure (i.e. `name`) to be validated again since otherwise, construction of `Person` would have failed. --- This refactor continues patching the method `_deserialize` of the schema fields but, instead of deciding how to patch it according to the type, it makes `_deserialize` to call a new custom method `_validate_model` with the desired behaviour. The different strategies from the if/elif/else block of the original implementation are now spread over the fields defined in qiskit.validation.fields This package also contains versions of marshmallow.fields that include their own `_validate_model` method. * Organize the fields into several modules and factor out validation. This PR split the fields into general fields, containers and polymorphic. Each module has the definition of the different kind of fields. It also provides a base field for those able of model type validation and a general mechanism to perform this validation. * Polishing docstrings and reviewing container and polymorphic validations. * Rename to make intentions clear and fixed fields hierarchy * Moving tests to its own folder * Split tests into different categories * Docstrings fixes and clarifications. * Style, convention and doc tweaks * Update CHANGELOG * Make to_dict and from_dict more discoverable, adjust converter tests (Qiskit#1428) * Move to_dict and from_dict to BaseModel Move the `to_dict()` and `from_dict()` methods directly to the `BaseModel` instead of appending them at decoration time, for ease of usage and aiding discovery. * Fix converter tests * Result postprocess (Qiskit#1404) * changelog * add result.postprocessing functions * format_statevector * restore ibmq _reorder_bits * update get_statevector, get_unitary, get_counts * remove the temporary hex conversion of tests * update remaining tests * Use integers in properties.gates.qubits (Qiskit#1427) Use `integer` instead of `number` in the schema for BackendProperties (`BackendProperties.gates[x].qubits`). * Cleaning up * Linting

delapuente requested review from 1ucian0, ajavadia, atilag, diego-plan9, ewinston, jaygambetta, mtreinish, nonhermitian and taalexander as code owners November 28, 2018 00:22

delapuente force-pushed the feature/refactor-shallow-schema branch from 56e0dc9 to 5c62445 Compare November 28, 2018 08:15

delapuente added 5 commits November 28, 2018 10:10

Polishing docstrings and reviewing container and polymorphic validati…

d0cc99c

…ons.

Rename to make intentions clear and fixed fields hierarchy

ea4b397

Moving tests to its own folder

f87ddc3

delapuente force-pushed the feature/refactor-shallow-schema branch from 5c62445 to 6647574 Compare November 28, 2018 09:12

Split tests into different categories

bc0e6e5

delapuente force-pushed the feature/refactor-shallow-schema branch from 6647574 to bc0e6e5 Compare November 28, 2018 09:17

delapuente changed the title ~~[WIP] Refactor shallow schema~~ Refactor shallow schema Nov 28, 2018

Docstrings fixes and clarifications.

caf2250

diego-plan9 self-assigned this Nov 29, 2018

diego-plan9 added 4 commits December 4, 2018 20:23

Merge remote-tracking branch 'qiskit/master' into feature/refactor-sh…

7173efd

…allow-schema

Style, convention and doc tweaks

9be2463

Update CHANGELOG

2f046e8

Merge branch 'master' into feature/refactor-shallow-schema

987cd18

atilag approved these changes Dec 5, 2018

View reviewed changes

diego-plan9 merged commit f91d20f into Qiskit:master Dec 5, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor shallow schema #1350

Refactor shallow schema #1350

delapuente commented Nov 28, 2018

Refactor shallow schema #1350

Refactor shallow schema #1350

Conversation

delapuente commented Nov 28, 2018