Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support custom template path #231

Merged
merged 14 commits into from
Nov 4, 2020

Conversation

erichulburd
Copy link
Contributor

@erichulburd erichulburd commented Nov 3, 2020

This is a WIP, just need to add tests, but before going down that road, I wanted to confirm repo owners would embrace this change.

Pretty well supported method of customization by the folks at Jinja.

TODO:

  • Ensure relative paths are respected if absolute path is not passed.
  • Add unit tests
  • Add integration test
  • Fix broken unit test assertions

Solves #202 (and potentially other issues) implicitly.

@erichulburd erichulburd force-pushed the accept_custom_templates branch from 63a2c85 to bbef8b3 Compare November 3, 2020 00:35
@dbanty
Copy link
Collaborator

dbanty commented Nov 3, 2020

This looks great! My main reason for not adding custom template support thus far was that our existing set of templates and handling thereof is pretty naive (mostly due to my own limited experience with jinja2). In a perfect world, we'll have a well thought out and well documented template "API" that authors of custom templates can reference when writing their own. This probably means a pretty significant refactor of the existing templates.

If we want to add this (mostly) as is, I think we just need a big disclaimer that the templating API is very unstable and will break at a later date, so use with care.

I think in it's final form we'll want features like:

  1. Better thought out and documented API, probably including customizable blocks as well
  2. Ability to specify template(s) from Git repo(s) for easier sharing/usage
  3. Ability to completely customize file generation? Right now which files are generated is pretty much hard coded, but that doesn't have to be the case. In the case of something like Add object-oriented client option #224 they might not want the same project structure as us. I don't know exactly what this would look like- probably a Python file which can contain an alternate Project implementation?

@erichulburd
Copy link
Contributor Author

I just added documentation for template usage in the Readme and included the following warning:

Be forewarned, this is a beta-level feature in the sense that the API exposed in the templates is undocumented and unstable.

I think you bring up good points, but I think a wild west templating API with a FileSystemLoader is sufficient for a preliminary release. Getting some early usage may help us color in the approach for settling the points you bring up.

Off cuff:

  1. Here it'd be nice to stay as close to the OAS schema as possible - if OAS data-types could be auto-generated themselves, that would be ideal. A cursory search for a tool we could leverage here would be: https://github.com/koxudaxi/datamodel-code-generator/. This leverages pydantic (which has some nice features beyond what dataclasses offer). Its name seems to imply its scope is limited to defining the data types; this repo could then just focus on implementing the actual API operation calls.
  2. Jinja doesn't support that right out of the box, but does expose a FunctionLoader which we could use to pull over the network. My hesitation here is, once you support git, or any other network call, you're opening yourself up to handling authentication, retries, etc. Seems like the responsibility to value proposition is high there, since any user with git access could just clone the repo and point to a file path.... (On the other hand, I acknowledge a network loader of some kind would more effectively abstract that clone/ download step, making single command generation easier).
  3. Once you're talking about importing a user defined Project implementation, I think what you're really talking about is exposing parts of the generator internals here that the user could import, subclass, and call in their own script (rather than calling through a CLI). It does seem like this would commit the repo to more responsibility by expanding the public API; whether that's worthwhile all depends on how valuable it would be for other users. It's not something my current use case motivates, but you would know better than me if other users would find it valuable!

A fully customizable template tree is interesting, but seems complicated in the sense that you would have to think through what variables do you expose to each template (eg imagine iterating through endpoints). Blocks could be useful here.

@erichulburd
Copy link
Contributor Author

@dbanty the above aside, let me know if there are specific TODOs that need to be checked off before this PR is ready for merge.

Copy link
Collaborator

@dbanty dbanty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple changes and this is good to go:

  1. Validation on passed in path to prevent potential unhandled exceptions
  2. Remove the checked in test-reports

@@ -100,6 +100,7 @@ def handle_errors(errors: Sequence[GeneratorError]) -> None:
def generate(
url: Optional[str] = typer.Option(None, help="A URL to read the JSON from"),
path: Optional[pathlib.Path] = typer.Option(None, help="A path to the JSON file"),
custom_template_path: Optional[str] = typer.Option(None, help="A path to a custom template directory"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does FileSystemLoader do in the event the directory doesn't exist or isn't a directory? We should probably take in a pathlib.Path here so that Typer can do some validation for us.

@erichulburd erichulburd requested a review from dbanty November 4, 2020 16:52
@erichulburd
Copy link
Contributor Author

@dbanty Done and done. Do you typically squash on merge? Or do we need to reorganize the commits here at all?

@dbanty dbanty added this to the 0.7.0 milestone Nov 4, 2020
Copy link
Collaborator

@dbanty dbanty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks! I squash PRs, so no need to fix up commits here. I'll include this in 0.7.0 with a few other changes I expect to be done by the end of this week.

@dbanty dbanty merged commit 3cd8a65 into openapi-generators:main Nov 4, 2020
@erichulburd
Copy link
Contributor Author

Sweet thanks @dbanty !

@erichulburd erichulburd deleted the accept_custom_templates branch November 4, 2020 17:22
dbanty added a commit that referenced this pull request Nov 4, 2020
@dbanty dbanty mentioned this pull request Nov 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants