Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python 3.7] Maven generation api 3.0.9 #9546

Closed
Badmiaoo opened this issue Jul 5, 2019 · 19 comments
Closed

[Python 3.7] Maven generation api 3.0.9 #9546

Badmiaoo opened this issue Jul 5, 2019 · 19 comments
Assignees

Comments

@Badmiaoo
Copy link

Badmiaoo commented Jul 5, 2019

Description

Hello,
I generate my APIs in python with maven. I use the version 3.0.8 of swagger-codegen-maven-plugin. But python is an experimental moud and i have some error about Thread can't be created (maybe their are not closed) and MemoryError. So i want to use the version 3.0.9, with hope to not find the same errors.

But impossible to build my APIs, see below:

POM.XML
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.eh</groupId>
    <version>1.3.1-SNAPSHOT</version>
    <artifactId>tci-apis</artifactId>
    <packaging>jar</packaging>
    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>
    <name>tci-apis</name>
    <build>
        <plugins>
            <!--&lt;!&ndash; Parse project version to get version identifiers &ndash;&gt;-->
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <version>1.8</version>
                <executions>
                    <execution>
                        <id>parse-version</id>
                        <goals>
                            <goal>parse-version</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <!-- Iterate over api specs and generate clientes -->
            <plugin>
                <groupId>com.soebes.maven.plugins</groupId>
                <artifactId>iterator-maven-plugin</artifactId>
                <version>0.5.1</version>
                <executions>
                    <!-- GENERATE SOURCES PHASE -->
                    <execution>
                        <phase>generate-sources</phase>
                        <id>generate-swagger-clients</id>
                        <goals>
                            <goal>iterator</goal>
                        </goals>
                        <configuration>
                            <items>toto,tata</items>
                            <pluginExecutors>
                                <pluginExecutor>
                                    <plugin>
                                        <groupId>io.swagger.codegen.v3</groupId>
                                        <artifactId>swagger-codegen-maven-plugin</artifactId>
                                        <version>3.0.9</version>
                                    </plugin>
                                    <goal>generate</goal>
                                    <configuration>
                                        <inputSpec>handlebars-swagger-specs/api-tci-@[email protected]</inputSpec>
                                        <output>clients/api_tci_@item@_client</output>
                                        <language>python</language>
                                        <addCompileSourceRoot>true</addCompileSourceRoot>
                                        <generateApis>true</generateApis>
                                        <generateApiDocumentation>true</generateApiDocumentation>
                                        <generateApiTests>true</generateApiTests>
                                        <generateModels>true</generateModels>
                                        <generateModelDocumentation>true</generateModelDocumentation>
                                        <generateModelTests>true</generateModelTests>
                                        <generateSupportingFiles>true</generateSupportingFiles>
                                        <configOptions>
                                            <packageName>@item@_api</packageName>
                                        </configOptions>
                                    </configuration>
                                </pluginExecutor>
                            </pluginExecutors>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>
COMMAND
mvn generate-sources
ERROR OUTPUT
Exception in thread "main" java.lang.StackOverflowError
        at java.util.HashMap.hash(Unknown Source)
	at java.util.LinkedHashMap.get(Unknown Source)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:246)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
...

Can you help me please ? :)

@chrisinmtown
Copy link

I suggest that you post your API spec so the developers can try to reproduce the problem.

@Badmiaoo
Copy link
Author

Sorry, but i can't. My company refuse to make the code public.

@derkoe
Copy link
Contributor

derkoe commented Jul 15, 2019

I have the same issue with 3.0.9 and 3.0.10 - 3.0.8 works. The issue seems to be in https://github.com/swagger-api/swagger-codegen-generators

@vinzent
Copy link

vinzent commented Jul 17, 2019

I've got the more or less same issue using the 3.0.10 swagger-codegen-cli docker image. It doesn't depend if i choose spring or python as target language. The OpenAPI3 spec i'm using is a converted OData CSDL from our internal CMDB. Rather largish - json file is 8mb. There are many linked resources (Computers have Adapters, Adapters have Computers, ...)

Unfortunatly I can't provide the OpenAPI3 spec and I failed to create a reproducer with Petstore. :-(

Generate completes successfully with swagger-codgen-cli 3.0.8 - but Python code is invalid (which i think is fixed with 3.0.10).

...
12:36:26.935 [Thread-1] DEBUG io.swagger.codegen.v3.generators.examples.ExampleGenerator - String property
12:36:26.935 [Thread-1] DEBUG io.swagger.codegen.v3.generators.examples.ExampleGenerator - No values found, using property name PowerState as example
12:36:26.935 [Thread-1] DEBUG io.swagger.codegen.v3.generators.examples.ExampleGenerator - String property
12:36:26.935 [Thread-1] DEBUG io.swagger.codegen.v3.generators.examples.ExampleGenerator - No values found, using property name Status as example
12:36:26.935 [Thread-1] DEBUG io.swagger.codegen.v3.generators.examples.ExampleGenerator - String property
12:36:26.935 [Thread-1] DEBUG io.swagger.codegen.v3.generators.examples.ExampleGenerator - No values found, using property name SupportOrganization as example
Exception in thread "Thread-1" java.lang.StackOverflowError
        at java.util.AbstractMap.toString(AbstractMap.java:547)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:162)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:194)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
....

@HugoMario HugoMario self-assigned this Jul 18, 2019
@HugoMario
Copy link
Contributor

hello guys, working on this right now.

@HugoMario
Copy link
Contributor

hey @Badmiaoo @vinzent i've been trying to reproduce this with no success yet. i understand you can not share the OAS 3 definition you're using, but please consider share a specific sample that generates the issue in order to have a better way to fix it.

@vinzent
Copy link

vinzent commented Jul 18, 2019

@HugoMario Could you provide a jar that prints out the schema and the property that the resolveSchemaToExample() (https://github.com/swagger-api/swagger-codegen-generators/blob/master/src/main/java/io/swagger/codegen/v3/generators/examples/ExampleGenerator.java#L160) currently is processing ?

I was able to build swagger-codgen with run-in-docker.sh but I have no clue how to integrate such a change from dependent repo.

@vinzent
Copy link

vinzent commented Jul 19, 2019

@HugoMario I came to the conclusion, that the problem is the recursive function:

my spec got 1527 schemas with 14820 properties. And everything is heavily linked.

@vinzent
Copy link

vinzent commented Jul 19, 2019

@HugoMario managed to convert the CSDL with OpenAPI.NET.OData to OpenAPI3.

swagger-codegen then exited successfully. :-)

but the client is still not useable. But this is another issue.

@HugoMario
Copy link
Contributor

HugoMario commented Jul 19, 2019

hi @vinzent, thanks for updates, checking issue i found a way to reproduce it too, in a scenario when we find circular references i.e:

    Category:
      type: object
      properties:
        id:
          type: integer
          format: int64
        pet:
          $ref: '#/components/schemas/Pet'

    Pet:
      type: object
      properties:
        id:
          type: integer
          format: int64
        category:
          $ref: '#/components/schemas/Category'

the issue is triggered, i guess it's the same if there are circular reference among three or more schemas.

@shathor
Copy link

shathor commented Jul 24, 2019

Similar issue that only occurs on version 3.0.9 and 3.0.10 but is fine on 3.0.8 (not using Python but Java but that shouldn't make a difference?):

Exception in thread "main" java.lang.StackOverflowError
        at java.lang.String.lastIndexOf(String.java:1827)
        at java.lang.String.lastIndexOf(String.java:1807)
        at io.swagger.codegen.v3.generators.util.OpenAPIUtil.getSimpleRef(OpenAPIUtil.java:33)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:231)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:240)
        at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)

@scottydawg
Copy link

I'm seeing the same thing (recursive resolveSchemaToExample invocations). There's a couple of confusing things here though:

  • The ExampleGenerator has no resolveSchemaToExample method in the latest 3.0 version of that class.
  • The package shown in the exception output doesn't match the actual package:
    • Expected: io.swagger.codegen.v3.examples
    • Actual (in the log): io.swagger.codegen.v3.generators.examples

I've compiled and installed all of the swagger-codegen projects locally, and I'm attempting to generate my schema in offline mode to ensure that I use the locally installed variants:

mvn clean generate-sources -o -Pjava

I have a couple of profiles for generating code in different languages, hence the -P.

@scottydawg
Copy link

Ah...never mind. I see that this error is from this ExampleGenerator in the swagger-codegen-generators module. I confused it with the ExampleGenerator in this module.

@scottydawg
Copy link

Here's a sample spec that causes this failure. It models a simple hierarchy of nodes of a single type.

openapi: 3.0.0
info:
  title: "Recursion Demo, Issue #9546"
  description: This schema includes structures that cause recursive failure in ExampleGenerator.
  version: "0.1.0"

paths:
  /recursion:
    get:
      summary: fail
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OptionHierarchyNode'

components:
  schemas:
    OptionHierarchyNode:
      type: object
      required:
        - key
      properties:
        key:
          type: string
          description: The key of an object (in a separate dictionary) describing the attributes of
            the node.
        defaultValue:
          type: string
          description: The key of the node in the options list that is selected by default.
        displayOrder:
          type: number
          format: integer
          description: Optional; the intended position of the node relative to sibling nodes. If
            undefined, the UI should display them in the order in which they are listed. Ordered
            nodes take precedence over unordered nodes.
        options:
          type: array
          description: Selectable values for the current node.
          items:
            $ref: '#/components/schemas/OptionHierarchyNode'

@scottydawg
Copy link

I added the following warning before this invocation (in the external module):

logger.warn("Recursively resolving schema for " + propName.toString());

and the schema above produced the following output, with the warnings repeated recursively, as expected, until the stack overflow:

[WARNING] Recursively resolving schema for key
[WARNING] Recursively resolving schema for defaultValue
[WARNING] Recursively resolving schema for displayOrder
[WARNING] Recursively resolving schema for options
[WARNING] Recursively resolving schema for key
[WARNING] Recursively resolving schema for defaultValue
[WARNING] Recursively resolving schema for displayOrder
[WARNING] Recursively resolving schema for options
...
---------------------------------------------------
Exception in thread "main" java.lang.StackOverflowError
	at sun.nio.cs.UTF_8.updatePositions(UTF_8.java:77)
	at sun.nio.cs.UTF_8.access$200(UTF_8.java:57)
	at sun.nio.cs.UTF_8$Encoder.encodeArrayLoop(UTF_8.java:636)
	at sun.nio.cs.UTF_8$Encoder.encodeLoop(UTF_8.java:691)
	at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:579)
	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:271)
	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
	at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
	at java.io.PrintStream.write(PrintStream.java:526)
	at java.io.PrintStream.print(PrintStream.java:583)
	at org.fusesource.jansi.FilterPrintStream.write(FilterPrintStream.java:99)
	at org.fusesource.jansi.FilterPrintStream.write(FilterPrintStream.java:107)
	at org.fusesource.jansi.FilterPrintStream.print(FilterPrintStream.java:156)
	at org.fusesource.jansi.FilterPrintStream.println(FilterPrintStream.java:231)
	at org.slf4j.impl.SimpleLogger.write(SimpleLogger.java:318)
	at org.slf4j.impl.SimpleLogger.log(SimpleLogger.java:295)
	at org.slf4j.impl.SimpleLogger.warn(SimpleLogger.java:522)
	at io.swagger.codegen.v3.generators.examples.ExampleGenerator.resolveSchemaToExample(ExampleGenerator.java:247)

@scottydawg
Copy link

@HugoMario - I've created a PR to address this in the swagger-codegen-generators repo.

@HugoMario
Copy link
Contributor

hi @scottydawg, yes, i saw your PR thanks a lot !!, i'll add feedback there

@HugoMario
Copy link
Contributor

hello guys PR's swagger-codegen-generators#431 and swagger-codegen-generators#432 have been added in order to fix this issue.

Can you please check if it works for you now?

you can use this file with command:

java -jar ./swagger-codegen-cli-3.0.11-20190725.145929-11.jar generate -i ./input-file.yaml -l python -o ./output

@scottydawg
Copy link

I successfully ran this against my schema without the stack overflow. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants