Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for the REST File format (Jetbrains, VSCode, Neovim) #205

Closed

Conversation

benfaerber
Copy link
Contributor

@benfaerber benfaerber commented May 4, 2024

Description

Add Support for the REST file format (Jetbrains, VSCode, Neovim)

Addresses Issue #122

This PR adds support for the REST file format.
In Jetbrains this file ends with .http and has some unique features not supported in this importer (running Javascript, dyanmic variables, etc)
In VSCode, Visual Studio, and Neovim this file ends with .rest.
I use this format quite often and I ported over all the features I use, plus some more. So I think this covers most of the general use cases. Super advanced features are not ported either because of complexity or security risks (running javascript).

Sorry, this is a large PR! A lot of the bulk is unit tests and the test files. I wanted it to be pretty usable before making a PR. This is one of those things where the whole parser has to be complete before it is usable. If the parser is out of scope, I could throw it into a new crate and then we could import from that. The crate would have the basic format parser and then the jetbrains env parser as an optional feature. The issue with that is there would have to be a lot of bridge code to convert types. It would also be annoying because you can't provide implementations for foreign types. I would like some thoughts on this.

Luckily, this project already had nom, serde, and url. I had to add httparse but that was already in the dependency tree via hyper.

Modules:

/collections/rest_format.rs

This module handles the parsing of the format and the conversion to a slumber collection.
It has some unit tests that cover most use cases but I'm sure I missed something. I ended up calling it rest_format instead of jetbrains because it works on a variety of http/rest file formats for several editors. I ported over only the basic features (requests, comments. variables, names) which makes it editor agnostic.

A basic overview of the parser:

  1. Read the file
  2. Split the file into lines
  3. Convert each line into a token, which is either a seperator, variable, name annotation or request. Build a map of variables and removes comments.
  4. Once the lines have been generated, merge them into raw http requests with optional names
  5. Parse the raw http requests using httparse and extract all the request info (headers, query, body, etc)
  6. One weird thing I had to do was create a "fake url" to parse the query. The url crate does not allow parsing of query alone so I had to append a localhost url to the query, then parse that. I did this because the template literals would cause an error in the URL parser. I thought it would be better to use url instead of hand rolling a query parser (sounds easy but I'm sure there are some edge cases I'd rather not handle)
  7. If there is an Authorize header, parse it and turn it into the slumber Authentication struct (supports Bearer and Basic)
  8. Using the collected recipes and variables convert to a collection (if it is a jetbrains env import, parse the jetbrains env and include it in the profiles)

/collections/jetbrains_env.rs

This module parses jetbrains http enviroment files.
This is a JSON file that is comparable to a slumber profile. There is a public and private variant.

Using the commands import jetbrains-with-public-env and import jetbrains-with-private-env you can include these files into your slumber collection.
These files are stored in the same directory as the HTTP file in jetbrains. The importer just searches the directory of the http file to find the envs.

/cli/import.rs

I added the following import targets:

  • vscode
  • jetbrains
  • jetbrains-with-public-env
  • jetbrains-with-private-env

I added the 3 jetbrains formats because it was the simplest way.
I think a custom Jetbrains env type flag might be better (ie import jetbrains --env private). Open to ideas.
I want to make sure the user knows when a private env is being imported as a lot of people wouldn't want that.

Known Risks

The file format is complex and some obscure use cases could cause errors but I have some good error messages.

QA

Unit tests for most functions.

Checklist

  • [x ] Have you read CONTRIBUTING.md already?
  • [ x] Did you update CHANGELOG.md?
    • Only user-facing changes belong in the changelog. Internal changes such as refactors should only be included if they'll impact users, e.g. via performance improvement.
  • [ x] Did you remove all TODOs?
    • If there are unresolved issues, please open a follow-on issue and link to it in a comment so future work can be tracked

@LucasPickering
Copy link
Owner

@benfaerber Do you think you'll have time to pick this back up? I think a first good step would be to break it into two PRs, one for the base .rest format then a second for the Jetbrains format.

@LucasPickering
Copy link
Owner

@benfaerber I'm closing this since it's inactive. If you'd like to keep working on it feel free to re-open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants