Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs to migrate REST APIs to Twirp #347

Merged
merged 4 commits into from
Feb 7, 2022
Merged

Docs to migrate REST APIs to Twirp #347

merged 4 commits into from
Feb 7, 2022

Conversation

marioizquierdo
Copy link
Contributor

Even though the migration process is a case-by-case scenario, it is still useful to have some basic guidelines for the recommended approach. If the community has useful tools that could be used in some cases, we could add links to those tools in this page too. If there are other guides for migrating from gRPC, Swagger and others, we could add those sections on this document later too.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

1. `cd website`
2. `npm install`
3. `npm start`
4. Navigate to http://localhost:3000/twirp.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only part I updated in this document. The way to install docusaurus changed a little bit and the docs no longer apply. Everything else was prettify in my editor "fixing" the markdown


Migrating existing REST/JSON APIs to Twirp is a case-by-case scenario. Unfortunately, it involves creating a new service, generating a new client, and migrating callers to use the new endpoints in the new client.

Twirp APIs are restricted to requests in the form `POST [<prefix>]/[<package>.]<Service>/<Method>` (see [Routing and Serialization](routing.md)). That limitation is what allows to build simple and consistent API schemas, but it also means that each REST API endpoint will have a different way to be translated into a Twirp endpoint. Most REST/JSON APIs don't have strongly defined schemas, and tend to have quirks and edge cases that are hard to automate.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change "what allows to build" to "what allows building"

2. Write a Protobuf schema with the same endpoints. Remember that Twirp endpoints depend on the Protobuf message used as request, but can also read [HTTP headers](headers.md) if needed. Respect Protobuf [naming and best practices](best_practices.md) when possible, but don't try and significantly change the API design of your application during the migration, it will be a lot easier to migrate callers to the new API if the new endpoints closely resemble the old endpoints. Try to use parameter types that are similar to the older parameters. Use comments on the schema to clarify edge cases.
3. If the API is too big, it may be better to only migrate a few endpoints first. Twirp APIs are good at evolving over time, you can always add more endpoints later.
4. Generate code and implement your new API methods. You may be able to re-use some of the older code from the previous endpoints. One way to do this is to implement the new service in a subfolder inside the same project, and then mount the new Twirp handler in a sub-route (you can use the default `/twirp` prefix, or specify a different prefix path if needed).
5. Make sure to track stats about API endpoint usage, so you can see the traffic being migrated to the new endpoints. Using [hooks or interceptors](hooks.md), or HTTP [middleware](mux.md) can be useful.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove comma in "), or HTTP"

3. If the API is too big, it may be better to only migrate a few endpoints first. Twirp APIs are good at evolving over time, you can always add more endpoints later.
4. Generate code and implement your new API methods. You may be able to re-use some of the older code from the previous endpoints. One way to do this is to implement the new service in a subfolder inside the same project, and then mount the new Twirp handler in a sub-route (you can use the default `/twirp` prefix, or specify a different prefix path if needed).
5. Make sure to track stats about API endpoint usage, so you can see the traffic being migrated to the new endpoints. Using [hooks or interceptors](hooks.md), or HTTP [middleware](mux.md) can be useful.
6. The API callers (upstream services) need to import the new Twirp client and update their calls to use the new Twirp API. Here is where making the new API similar to the old one will be the most useful. Some services may be able to just replace the old client with the new one and start sending traffic to the new endpoints. If they have a decent staging/canary environment and a good deploy system that allows for quick rollbacks, that could be enough to test that the new API is handling the new traffic properly. In other cases, you will need to implement a slider to migrate traffic slowly (10%, 50%, 100%). The slider can be implemented in different ways, either with if statements, e.g. `if r.Intn(100) < 10 { callNewClient() } else { callOldClient() }`, or in more complex cases, with a custom client that implements the same Twirp service interface, but it is able to split the traffic between the old API and the new API.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change "Here is where making the new API similar to the old one will be the most useful." to "This is why making the new API similar to the old one is useful."

@marioizquierdo
Copy link
Contributor Author

@wmatveyenko thanks for the review. I addressed the docs with the suggestions, and took the chance to give it another edit pass.

@wmatveyenko wmatveyenko merged commit af2c451 into main Feb 7, 2022
@wmatveyenko wmatveyenko deleted the docs_rest_to_api branch February 7, 2022 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants