-
-
Notifications
You must be signed in to change notification settings - Fork 6.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ruby] fix oneOf handling #5706
Conversation
👍 Thanks for opening this issue! The team will review the labels and make any necessary changes. |
Does this PR address the below part in the original bug?
|
@gommo ah right, that's not addressed. Even the java generator with improved oneOf handling fails in this case. In both languages it refers to some "EndpointGET200" model that doesn't exist. @bkabrda is this a known issue of the current implementation or should I open a new issue specifically for this? |
Better open an issue with details (e.g. mininal spec) to track the issue. |
@jfeltesse-mdsol thanks for the PR. When you've time, can you please PM me via Slack to discuss this PR? |
any chance to get this reviewed/merged? |
Tested with a spec with an oneOf schema containing null:
Something that we need to address later (not this PR) to properly handle nullable oneOf schemas. |
I did a test with primary types in oneOf schemas and looks like your current approach assumes the schemas defined in oneOf are models (with properties). Something we need to handle/address in future PRs. |
If
I don't think this will work as the Example definition:
|
Are you referring to the case where the oneOf members are other types like array or scalars? How are the other languages handling this? |
Example:
Java (jersey2), Go, PowerShell generator should be able to handle this without issues. |
If I understand correctly, your current approach relies on |
Actually it does work without a discriminator, the
|
That's good to know.
Should it go through the rest to ensure there's only one match (to fulfill the definitions of |
For the spec used by the Ruby petstore samples listed below, can you please switch it to |
I don't think so, the logic I described is used when deserizalizing the response and doing validation at that stage brings no benefit. On the request side you wouldn't use the module directly but rather pass the relevant data directly. There, alongside the existing client side validations, checking the oneOf promise may make sense but it should be done in a separate PR to keep things bite-sized. |
So |
Correct. As a user though, I have a hard time picturing how validation when receiving can be useful. Imagine I'm using an autogenerated client to access an external API I have no control over. What good does it do to me that even though I received some data, the client throws an error (and prevents me from accessing the data?) because the API's openapi spec file incorrectly advertises oneOf when it really should be anyOf? Having said that though, it's not a huge challenge to validate that part. How the client should behave when encountering such cases though? Throw an error? Return null? Return the first valid item and log a warning? some other way?? |
The user can then report the issue to the API owner to look into the error as the response (payload) doesn't conform to the contract/spec (
Sorry that it's not something I can agree with. Both client and server should conform to the contract (openapi spec).
In Java, Go, PowerShell implementation, the client throws an exception or returns an error. |
{{/discriminator}} | ||
{{^discriminator}} | ||
openapi_one_of.each do |_class| | ||
model = {{moduleName}}.const_get(_class).build_from_hash(attributes) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be wrapped in a try/rescue. build_from_hash
for enum values can raise. And when that's the case, the loop should continue to try the next model.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
build_from_hash for enum values can raise
can you provide an example?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh one more thing I noticed, as I'm trying out this PR on our spec: the oneOf
types can be primitive types like Float
, String
, ... so we can't call build_from_hash
on those. Maybe you could add some special cases, comparing the openapi_one_of
value to primitive types, and try to directly parse the value in that case ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this has been pointed out and it is something I plan to address #5706 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right, sorry for the noise 🙈
heads up here, I've not abandoned this PR, it's just that I got super busy at work. I might be able to go back to this in a couple of weeks. |
@jfeltesse-mdsol please take your time. We appreciate your contributions to this. |
@wing328 About #5706 (comment), I rebased my branch with latest master, re-built the generator and updated the inputSpec with --- a/bin/configs/ruby-faraday.yaml
+++ b/bin/configs/ruby-faraday.yaml
@@ -1,7 +1,7 @@
generatorName: ruby
outputDir: samples/client/petstore/ruby-faraday
library: faraday
-inputSpec: modules/openapi-generator/src/test/resources/3_0/petstore-with-fake-endpoints-models-for-testing.yaml
+inputSpec: modules/openapi-generator/src/test/resources/3_0/java/petstore-with-fake-endpoints-models-for-testing-with-http-signature.yaml
templateDir: modules/openapi-generator/src/main/resources/ruby-client
additionalProperties:
gemVersion: 1.0.0 But then running
Any clue? The input file doesn't have any |
@wing328 About the logic and ensuring the oneOf promise is fullfilled, how about verifying it only when a discriminator is not set? This is to conform to the openapi spec's explanation about discriminator:
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#discriminator-object IMHO, if we always validate all the schemas then it makes the discriminator property pointless and undermines the work of the API spec author. |
modules/openapi-generator/src/main/resources/ruby-client/model_test.mustache
Show resolved
Hide resolved
@wing328 So I had a look at validating the models that may be defined in the oneOf array so that when looping through those we can double check the oneOf promise. The take away is this is not as simple as it sounds and it should be done in a different PR. Basically, a bare bones implementation may look like {{^discriminator}}
valid_models = []
openapi_one_of.each do |_class|
model = {{moduleName}}.const_get(_class).build_from_hash(attributes)
valid_models << model if model.valid?
end
return valid_models[0] if valid_models.size == 1
raise "{{classname}} is using oneOf but more than 1 schema validated #{attributes} => #{valid_models.inspect}"
{{/discriminator}} Looks good right? Except the ObjA:
type: object
properties:
realtype:
type: string
message:
type: string
ObjB:
type: object
properties:
realtype:
type: string
code:
type: integer
ObjC:
type: object
properties:
realtype:
type: string
code:
type: string
ObjAOrObjB:
oneOf:
- $ref: '#/components/schemas/ObjA'
- $ref: '#/components/schemas/ObjB'
- $ref: '#/components/schemas/ObjC' In this case, that's because the So yeah, not opening that pandora's box just now. I'll try to complete the PR with support for scalar values though, stay tuned! |
Unless the CI reveals bugs, I think this is feature complete. The most interesting bits are in Now oneOf supports scalars, arrays, hashes and the combinations you may think of. Maybe not perfect but should support most of the cases. And yeah, it doesn't try to check the oneOf promise. I got it to actually check whether the properties passed existed (see the changes in base_object, before it'd build_from_hash without passing the data to the constructor, bypassing the property name checks...) but the values in base_object are cast without checking the type. If someone's pumped about fixing this, it should be the topic of a separate PR. I can't fix all the issues in one PR 😅 |
@wing328 I tried to update the default ruby version that's incorrect here |
Travis CI reported the following errors:
Ref: https://travis-ci.org/github/OpenAPITools/openapi-generator/builds/739218479 Please take a look when you've time. |
CircleCI, which runs the ensure-up-to-date script, didn't report any failure so I assume it's no longer an issue, right? |
@wing328 I'll have a look at the Travis failure tomorrow. As for the Circle CI one, it's still a problem, I reverted the change so it passes, that's it. The doc here is still going to advertise the default ruby version for generated clients as ">= 1.9" when in fact it's ">= 2.4". How to update the doc and make the CI not change it back? |
One way is to run |
Not sure if I explain this correctly but locally the script will always revert to the previous value. I commit this: commit ab351a70fb (HEAD -> ruby/fix_oneof)
Author: Julien Feltesse <[email protected]>
Date: Wed Oct 28 15:34:55 2020 +0900
use ruby 2.4 by default
diff --git a/docs/generators/ruby.md b/docs/generators/ruby.md
index 4c1cbca4e2..24b340eeec 100644
--- a/docs/generators/ruby.md
+++ b/docs/generators/ruby.md
@@ -16,7 +16,7 @@
|gemHomepage|gem homepage. | |http://org.openapitools|
|gemLicense|gem license. | |unlicense|
|gemName|gem name (convention: underscore_case).| |openapi_client|
-|gemRequiredRubyVersion|gem required Ruby version. | |>= 1.9|
+|gemRequiredRubyVersion|gem required Ruby version. | |>= 2.4|
|gemSummary|gem summary. | |A ruby wrapper for the REST APIs|
|gemVersion|gem version.| |1.0.0|
|hideGenerationTimestamp|Hides the generation timestamp when files are generated.| |true| Run the script:
And I get that diff: diff --git a/docs/generators/ruby.md b/docs/generators/ruby.md
index 24b340eeec..4c1cbca4e2 100644
--- a/docs/generators/ruby.md
+++ b/docs/generators/ruby.md
@@ -16,7 +16,7 @@
|gemHomepage|gem homepage. | |http://org.openapitools|
|gemLicense|gem license. | |unlicense|
|gemName|gem name (convention: underscore_case).| |openapi_client|
-|gemRequiredRubyVersion|gem required Ruby version. | |>= 2.4|
+|gemRequiredRubyVersion|gem required Ruby version. | |>= 1.9|
|gemSummary|gem summary. | |A ruby wrapper for the REST APIs|
|gemVersion|gem version.| |1.0.0|
|hideGenerationTimestamp|Hides the generation timestamp when files are generated.| |true| Where the heck the |
The doc is auto-generated. We'll take care of it in a separate PR instead. |
As mentioned in the last commit, I reverted to the old configs because the new ones would run into the issue described at #4690 |
modules/openapi-generator/src/main/resources/ruby-client/base_object.mustache
Show resolved
Hide resolved
|
fails on the .net things which I'm pretty sure aren't related to the PR here... 😑 |
@wing328 This PR is ready to be merged but some CIs are failing randomly. I tried to force-push a couple of times but it's one CI failing once, another CI failing afterwards, and so on. On unrelated stuff. |
hey @wing328, any chance this can get merged in soon ? We're excited to start using this feature for generating our ruby client. |
We were busy with the v5.0.0-beta3 release: https://twitter.com/oas_generator/status/1329828788264321025. Let me try to review today or tomorrow. Is it correct to say that you've tested the change locally and it works in your use cases? |
Yeah it works for the few use cases I tried |
@zippolyte thanks for testing it. I did a quick look and the PR looks good overall. I will merge this instead of being the bottleneck to get this PR merged. Will open an issue or file a PR if I find anything later. |
Just found a minor bug in the generated tests. They fail for oneOf containing primitive types because of the generated following code it 'lists the models referenced in the oneOf array' do
expect(described_class.openapi_one_of).to_not be_empty
described_class.openapi_one_of.each { |klass| expect { DatadogAPIClient::V2.const_get(klass) }.to_not raise_error }
end Primitive types are not constants of the API client module. |
ah, probably an oversight of mine when I refactored the code to accept scalar types and not just schemas. I think removing the 2nd line is ok since we wouldn't be able to make advanced tests in the generated code (lack of context). "Real" oneOf specs should be added in the samples but I didn't do that since the config need to be updated first. |
Fixes #3630.
I have yet to update the documentation related templates and update the samples but opening now to gather initial feedback on the direction.👌Basically, this builds up on the work from @bkabrda on the java and go experimental clients.not doing this in the end.I didn't go "ruby-experiemental" because the support of oneOf in the current ruby client is broken anyway.
How it works is, in the spirit of the "oneof as interfaces" this creates a module, not a class, for oneOf models and provides a set of class methods related to the functionality.
The core method is
build_from_hash
whose job in this case is to find the relevant oneOf item and transparently building and returning the correct model.It does so by using the discriminator if present, mapping if present and "falls back" to the existing validation-based look up.
I changed some test yaml files to include more oneOf examples.no need in the end.If I try to do a before/after using the modified
composed-one.yaml
file, it goes like (some parts trimmed for brevity):openapi-generator generate -i composed-oneof.yaml -g ruby --library faraday -o ./out/ruby_oneof
model files
custom_one_of_schema.rb
Noteworthy is in the obj_c.rb the nested oneOf property has its model generated, albeit with a sligthly different name:
Limitations / bugs / food for thought
custom_one_of_array_schema_one_of.rb
"interface" module is now generated but the oneOf is taking place inside theitems
. I'm not sure this solution is going to work here. In this case maybe a regular model should be generated alongside somecustom_one_of_array_schema_one_of_items.rb
"interface" module?@bkabrda how do the java and experimental go clients fare in this case?
PR checklist
./bin/
(or Windows batch scripts under.\bin\windows
) to update Petstore samples related to your fix. This is important, as CI jobs will verify all generator outputs of your HEAD commit, and these must match the expectations made by your contribution. You only need to run./bin/{LANG}-petstore.sh
,./bin/openapi3/{LANG}-petstore.sh
if updating the code or mustache templates for a language ({LANG}
) (e.g. php, ruby, python, etc).master
,4.3.x
,5.0.x
. Default:master
.@cliffano (2017/07) @zlx (2017/09) @autopp (2019/02)