Work towards OpenAPI 3.1 support #22
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@kevindew great to see there's progress in 3.1 support.. what the current status here? happy to support! |
rngtng
reviewed
Jun 29, 2023
| - perhaps have context support a merge concept for source location | ||
| - Think about dealing with recursive defined as "#" | ||
|
|
||
| Dealing with the new JSON Schema approach for OpenAPI 3.1. |
There was a problem hiding this comment.
what about piggyback on existing implementation like https://github.com/voxpupuli/json-schema?
There was a problem hiding this comment.
I'm a bit rusty in memory but there's some tricky challenges as I recall. If I remember right, that gem doesn't support a version of JSON schema as new as what OpenAPI supports and I have recollections that it's more useful for validation than it is for traversal.
|
@rngtng Hey, thanks for the interest and sorry for taking a while to get back to you. Currently I'm hoping to find some time in the next few months to pick this up again and make progress. It's generally a case of working through the items in https://github.com/kevindew/openapi3_parser/blob/fc98338bcc16fb724c3974eec8103ee76e61fb65/TODO.md |
|
thx @kevindew - let's see if I find time to get a deeper view and maybe support you.. |
|
@kevindew Hi. I suggest to focus on reading only and leave validation to a separate gem: https://github.com/davishmcclurg/json_schemer |
|
Is there any plan to resume support for version 3.1? |
|
Yup - it's definitely on my todo list but has been a real struggle to find time for working on this - going to try find some more time this year to progress. |
kevindew
force-pushed
the
openapi-3.1
branch
2 times, most recently
from
a833ca5 to
9d3e063
Compare
These are intended to be used as ways to determine whether particular OpenAPI features will be enabled.
This class is to replace the use of a string as the means for
representing the version of an OpenAPI version. A new class has been
added to make it simpler to compare versions in a more sophisticated way
than strings. The class inherits from Gem::Version which is a part of
Ruby stdlib which has this comparison logic built in.
The spaceship operator method has been replaced with one that will
apply polymorphish to comparisons. E.g:
It replaces the need for:
```
OpenapiVersion.new("3.11") > OpenapiVersion.new("3.2")
```
with:
```
OpenapiVersion.new("3.11") > "3.2"
```
I added this in as it felt more natural to do this than have some
contrived `equal_or_greater_than?` methods. However I do have a concern
that I may have created some Ruby magic
Previously this argument only accepted boolean arguments and didn't allow you to specify that this could execute code. This code changes that by allowing lambda functions and symbols (that reference methods) as arguments. This has been added so that specifying whether a field is required can be determined by the openapi_version.
As of OpenAPI 3.1 paths is no longer required [1] [1]: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#openapi-object
This is a new field that has been added in OpenAPI 3.1 [1] [1]: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#oasWebhooks
This feature allows fields of a node to be marked as whether they are allowed to exist or not. This has been added in to support the OpenAPI 3.1 where there are fields that are only valid in OpenAPI 3.1 documents. A field can take an allowed value of a boolean, a proc or a symbol. It is expected that, outside of contrived tests, it will never use a plain boolean as there is no point marking a field as not allowed everywhere.
This field was introduced with OpenAPI 3.1
A change introduced in OpenAPI 3.1 is that the responses field is no longer required [1] [1]: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#operation-object
This is a requirement from OpenAPI v3.1: The OpenAPI document MUST contain at least one paths field, a components field or a webhooks field. [1] [1]: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#openapi-document
This is an attempt to try catalogue the changes that are needed - the most substantial of which seem to be in the Schema object.
With OpenAPI 3.1 the schema object is quite different (conforming to 2012-12 spec of JSON Schema). I think the easiest way to deal with this is to create separate classes for working with it. This starts this process by creating Schema::V3_0 classes which is a rename of the existing Schema classes.
This is unfortunately rather long, but there doesn't seem any clear single responsibility opportunities that can be embraced so I'm taking the coward/assertive route of telling the linter: no.
In OpenAPI 3.1 references can have summary and description metadata for nodes that support those fields. They operate in a cascading pattern where each reference can overwrite the previous ones.
This adds the identifier field to License - as outlined in the OpenAPI 3.1 Specification [1]. A couple of things I considered doing for this, but ultimately didn't do, were: - Determine whether the SPDX license expression format could be validated with a regex. It looked like it probably could but not sure if we want to be that strict. - Add a hook in that would allow us to decide which OpenAPI versions would use the mutually_exclusive DSL method. I decided this wasn't necessary for now since the only benefit seems to be a slightly nicer error message.
We already validate this. In 3.0.x to 3.1.x the spec changed from: > An enumeration of string values to be used if the substitution options are from a limited set. The array SHOULD NOT be empty. to: > An enumeration of string values to be used if the substitution options are from a limited set. The array MUST NOT be empty.
This code doesn't validate the type of a Security Scheme so we don't need to change the code for the new type of mutualTLS.
A change in OpenAPI 3.1 is that the Discriminator object can accept extensions, when previously this was not allowed. In order to code this I've had to adjust the DSL so that the allow_extensions method can accept a block. We didn't have unit tests for the DSL so I'm only testing this at the node level. This should be ok as it has full coverage.
This resolves an error that is caused by the removal of `#sort`
from Dir.glob. Dir.glob produces a slightly different file ordering
without sort and thus causes the below missing constant error.
```
Failure/Error:
def allow_extensions(regex: EXTENSION_REGEX, &block)
@extension_regex = regex
@allowed_extensions = block || true
end
NameError:
uninitialized constant Openapi3Parser::NodeFactory::ObjectFactory::Dsl::EXTENSION_REGEX
def allow_extensions(regex: EXTENSION_REGEX, &block)
```
This sets up the ability for shared code between Schema objects and providing customisation for ones of different OpenAPI versions. It marks a direction towards modelling the specification of JSON Schema 2020-12 JSON Schema Validation [1] specifically rather trying to understand how different dialects could be used. My expectation is that it is very much an edge case that someone would use another dialect and it seems incredibly hard to consider how they can be supported. There's likely some optimising that could be done of the shared example specs, but I wanted to get something together on the faster side. This also has a rename of Schema::OasDialect3_1 to Schema::V3_1. I don't think we'll ever try to model other dialects so I think it's best to have a single class to try model the schemas for 3.1 and hopefully later versions. [1]: https://json-schema.org/draft/2020-12/draft-bhutton-json-schema-validation-01 WIP
This field is rather complicated as JSON Schema 2020-12 allows it to be an array or a string and other parts of OpenAPI don't allow different types. Another difference is that JSON Schema specification doesn't have the rule (or I didn't find it) where you require items to be specified if type is array. There are different types of items (such as prefixItems and additionalItems) in JSON Schema so I guess the items rule is complex.
I didn't think I'd be able to use these words as method names as they are Ruby reserved keywords. However as I understand it is ok because Ruby can work with them contextual (i.e. they're always prefixed by an object e.g self.if) Spec used: https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-00#section-10.2.2.1
Based on definition from [1] where it's a list of schemas that defaults to an empty array [1]: https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-00#section-10.3.1.1
This fixes an error in OpenAPI 3.0 schema handling where minimum and maximum were flagged as integers when actually they are numeric. It then corrects the configuration of exclusiveMaximum and exclusiveMinimum for OpenAPI 3.1 where JSON Schema 2021 [1] defines these as numeric values compared to the boolean values they held in OpenAPI 3.0 (via JSON Schema 2017 [2]). [1]: https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-6.2.3 [2]: https://datatracker.ietf.org/doc/html/draft-wright-json-schema-validation-00#section-5.3
Based on JSON schema 2021 [1]. Where the value is expected to be a map where the key is a regex (to match a string property name) and value is a Schema object. Like the implementation of pattern in this codebase this doesn't have validation that pattern is a regex, which could be added. [1]: https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-00#section-10.3.1.1
Something to think about adding soon
This is to fix minor errors that occur if a class other than hashes are used in these places. In these situations we were checking if objects responded to [] before using the method with a string key. However there are a bunch of objects that respond to [] and error if you provide a string key. E.g: ``` irb(main):001:0> arr = [] => [] irb(main):002:0> arr["$ref"] (irb):2:in `[]': no implicit conversion of String into Integer (TypeError) from (irb):2:in `<main>' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli/console.rb:19:in `run' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli.rb:516:in `console' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli.rb:31:in `dispatch' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli.rb:25:in `start' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/gems/3.1.0/gems/bundler-2.3.27/libexec/bundle:48:in `block in <top (required)>' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/friendly_errors.rb:120:in `with_friendly_errors' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/gems/3.1.0/gems/bundler-2.3.27/libexec/bundle:36:in `<top (required)>' from /Users/kevin.dew/.rbenv/versions/3.1.5/bin/bundle:25:in `load' from /Users/kevin.dew/.rbenv/versions/3.1.5/bin/bundle:25:in `<main>' irb(main):005:0> int = 25 => 25 irb(main):006:0> int["$ref"] (irb):6:in `[]': no implicit conversion of String into Integer (TypeError) from (irb):6:in `<main>' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli/console.rb:19:in `run' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli.rb:516:in `console' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli.rb:31:in `dispatch' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/cli.rb:25:in `start' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/gems/3.1.0/gems/bundler-2.3.27/libexec/bundle:48:in `block in <top (required)>' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/3.1.0/bundler/friendly_errors.rb:120:in `with_friendly_errors' from /Users/kevin.dew/.rbenv/versions/3.1.5/lib/ruby/gems/3.1.0/gems/bundler-2.3.27/libexec/bundle:36:in `<top (required)>' from /Users/kevin.dew/.rbenv/versions/3.1.5/bin/bundle:25:in `load' from /Users/kevin.dew/.rbenv/versions/3.1.5/bin/bundle:25:in `<main>' ``` I think there's very low or hopefully non existent chance that we'd be dealing with a hash like object in these scenarios so it seems safe to me to do the explicit type checking.
JsonSchema 2020-12 has a rather complex quirk that providing a boolean is treated as a valid schema. Where a value of `true` means the schema will pass anything and a value of `false` means the schema will pass nothing [1]. This is quite a pain to model as there is an assumption in this libraries code that a node can only be one type and that all references resolve to a map type. This code configures the factory to produce a v3_1 schema node when given these, in a subsequent commit I will configure the node class to make use of these. [1]: https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-00#section-4.3.2
These build on the smell of hacks from the previous commit to make this class a bit smelly. This adds some work arounds to get appropriate error messages for nodes that could be one of two types. Should we need more usage of this we probably want to refactor aspects of the ObjectFactory code to be able to handle dual types and the TypeChecker.
This will be modelled differently for OpenAPI 3.1 where a JSON schema can itself have a boolean value and thus we won't be using Boolean or Schema anymore as a 3.1 Schema will suffice.
I wasn't sure whether to add methods to the existing class or to create a new class. I'll see how this pans out.
…dProperties These schema properties return schemas but are commonly used as booleans. Therefore I've added boolean helper methods for them.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.