Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failures from openapi-schema-validator 0.4.2 #495

Closed
musicinmybrain opened this issue Feb 1, 2023 · 4 comments · Fixed by #498
Closed

Test failures from openapi-schema-validator 0.4.2 #495

musicinmybrain opened this issue Feb 1, 2023 · 4 comments · Fixed by #498

Comments

@musicinmybrain
Copy link
Contributor 

$ gh repo clone p1c2u/openapi-core
$ cd openapi-core
$ python3 -m venv _e
$ . _e/bin/activate
(_e) $ pip install poetry
(_e) $ poetry install
(_e) $ pytest

The tests pass.

Now:

(_e) $ pip install --upgrade openapi-schema-validator openapi-spec-validator
(_e) $ pytest

============================================== short test summary info ===============================================
FAILED tests/unit/unmarshalling/test_unmarshal.py::TestOAS30SchemaUnmarshallerCall::test_string_format_datetime_invalid - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_date_invalid[true] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_uuid_invalid[true0] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[true0] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[true1] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[1989-01-02] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_strict_rfc3339[1989-01-02T00:00:00Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/python...
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_strict_rfc3339[2018-01-02T23:59:59Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/python...
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_isodate[1989-01-02T00:00:00Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/python...
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_isodate[2018-01-02T23:59:59Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/python...
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_get_no_required_param - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_get_valid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_post_server_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_post_required_header_param_missing - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_post_media_type_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_post_required_cookie_param_missing - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_post_valid[data_json0] - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetListView::test_post_valid[data_json1] - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetDetailView::test_get_server_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetDetailView::test_get_unauthorized - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetDetailView::test_delete_method_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestPetDetailView::test_get_valid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/django/test_django_project.py::TestDRFPetListView::test_post_valid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_get_no_required_param - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_get_valid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_post_server_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_post_required_header_param_missing - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_post_media_type_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_post_required_cookie_param_missing - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_post_valid[data_json0] - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetListResource::test_post_valid[data_json1] - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetDetailResource::test_get_server_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetDetailResource::test_get_path_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetDetailResource::test_get_unauthorized - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetDetailResource::test_get_valid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/contrib/falcon/test_falcon_project.py::TestPetDetailResource::test_delete_method_invalid - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/schema/test_spec.py::TestPetstore::test_spec - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_response - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_response_no_schema - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_invalid_response - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_ids_param - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_tags_param - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_parameter_deserialization_error - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_wrong_parameter_type - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_raises_missing_required_param - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_empty_value - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_allow_empty_value - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_none_value - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_param_order - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pets_param_coordinates - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_birds - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_cats - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_cats_boolean_string - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_no_one_of_schema - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_cats_only_required_body - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_pets_raises_invalid_mimetype - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_pets_missing_cookie - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_pets_missing_header - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_pets_raises_invalid_server_error - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pet - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pet_not_found - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_pet_wildcard - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_get_tags - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_extra_body_properties - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_empty_body - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_wrong_property_type - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_additional_properties - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_created_now - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_created_datetime - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_created_invalid_type - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_delete_tags_with_requestbody - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_delete_tags_no_requestbody - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_petstore.py::TestPetstore::test_delete_tags_raises_missing_required_response_header - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_request_server_error - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_invalid_path - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_invalid_operation - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_missing_parameter - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_get_pets - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_get_pets_webob - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_missing_body - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_invalid_content_type - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_invalid_complex_parameter - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_post_pets - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_post_pets_plain_no_schema - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_get_pet_unauthorized - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestRequestValidator::test_get_pet - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_server - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_operation - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_response - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_content_type - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_missing_body - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_media_type - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_media_type_value - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_value - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_invalid_header - RecursionError: maximum recursion depth exceeded in comparison
ERROR tests/integration/validation/test_validators.py::TestResponseValidator::test_get_pets - RecursionError: maximum recursion depth exceeded in comparison
============= 10 failed, 677 passed, 5 xfailed, 2 xpassed, 277 warnings, 87 errors in 125.70s (0:02:05) ==============

(The full output is too long to paste here.)
@p1c2u
Copy link
Collaborator

p1c2u commented Feb 2, 2023

The reason is the issue here python-openapi/openapi-spec-validator#192

@p1c2u
Copy link
Collaborator

p1c2u commented Feb 2, 2023

The issue was fixed and new openapi-spec-validator version 0.5.4 released hence closing

@p1c2u p1c2u closed this as completed Feb 2, 2023
@musicinmybrain
Copy link
Contributor Author

The steps in this report still produce test failures (albeit fewer of them) even with openapi-spec-validator 0.5.4.

============================================ short test summary info =============================================
FAILED tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_created_now - openapi_core.validation.request.exceptions.InvalidRequestBody: Request body error
FAILED tests/integration/validation/test_petstore.py::TestPetstore::test_post_tags_created_invalid_type - assert <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaFormatValue'> is InvalidSchemaValue
FAILED tests/unit/unmarshalling/test_unmarshal.py::TestOAS30SchemaUnmarshallerCall::test_string_format_datetime_invalid - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_date_invalid[true] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_uuid_invalid[true0] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[true0] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[true1] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[1989-01-02] - Failed: DID NOT RAISE <class 'openapi_core.unmarshalling.schemas.exceptions.InvalidSchemaValue'>
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_strict_rfc3339[1989-01-02T00:00:00Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/py...
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_strict_rfc3339[2018-01-02T23:59:59Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/py...
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_isodate[1989-01-02T00:00:00Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/py...
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_isodate[2018-01-02T23:59:59Z] - AttributeError: <module 'openapi_schema_validator._format' from '/home/ben/src/forks/openapi-core/_e/lib64/py...
======================= 12 failed, 762 passed, 5 xfailed, 2 xpassed, 842 warnings in 4.97s =======================

If it matters, the above output is from Python 3.11.1.

@musicinmybrain
Copy link
Contributor Author

musicinmybrain commented Feb 2, 2023 •
edited
Loading

The steps in this report still produce test failures (albeit fewer of them) even with openapi-spec-validator 0.5.4.

I noticed that openapi-schema-validator 0.4.3 is not yet on PyPI. If I install it manually, I’m down to 7 failures:

=========================== short test summary info ============================
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_date_invalid[true]
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_uuid_invalid[true0]
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_invalid[true0]
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_strict_rfc3339[1989-01-02T00:00:00Z]
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_strict_rfc3339[2018-01-02T23:59:59Z]
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_isodate[1989-01-02T00:00:00Z]
FAILED tests/unit/unmarshalling/test_validate.py::TestSchemaValidate::test_string_format_datetime_isodate[2018-01-02T23:59:59Z]
====== 7 failed, 728 passed, 5 xfailed, 2 xpassed, 980 warnings in 2.02s =======

@p1c2u p1c2u reopened this Feb 3, 2023
@p1c2u p1c2u mentioned this issue Feb 3, 2023
Merged
@p1c2u p1c2u closed this as completed in #498 Feb 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

openapi packages update tests fix python-openapi/openapi-core
2 participants
@p1c2u @musicinmybrain