Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reStructured parser #67

Closed
plannigan opened this issue Oct 10, 2020 · 5 comments
Closed

Add reStructured parser #67

plannigan opened this issue Oct 10, 2020 · 5 comments
Labels
feature New feature or request

Comments

@plannigan
Copy link
Contributor

Is your feature request related to a problem? Please describe.
I have some projects that uses reStructured style of docstrings. I'd like to use mkdocstrings for these projects, but currently the google style is the only supported style supported.

Describe the solution you'd like
A second parser implementation that supports the directives documented on the Sphinx.

Describe alternatives you've considered
I found this blog post. However, it seems like multiple hoops to jump through when I'm already using mkdocs for the rest of the documentation for the these projects.

Additional context
I might be able to help with the implementation. I'd need to spend some time reading though the google parse to see how easy it would be to add a second parser.

@plannigan plannigan added the feature New feature or request label Oct 10, 2020
@pawamoy
Copy link
Member

pawamoy commented Oct 10, 2020

Hello! Maybe I can help you get started with an explanation of the current code.

A docstring parser is a class inheriting from pytkdocs.parsers.docstrings.base.Parser, accepting arguments at initialization (parsing options, up to you). It must implement a parse_sections method accepting a docstring as first argument. The parse_sections method must return a list of Section. You'll have access to the self.error(message) method to record a parsing error. You'll also have access to self.context which can contain various additional data supposed to help parsing. It will always contain an obj key: the reference to the object for which the docstring is written. In some cases, it will contain an attributes key as well:

{
    "attr1": {"docstring": "string": "annotation": int},
}

The rest is completely up to you! You can of course get inspiration from the Google style parser, though it's not the best looking or best structured code I've written.


⚠️ Note that even if pytkdocs can parse RST docstrings, mkdocstrings will still render them as if they were Markdown. It means that if you really want to keep using RST to write your docstring and make sure they are rendered correctly by mkdocstrings, you'll have to actually modify the docstrings somehow. I don't think I want this docstring-transformation code to land in pytkdocs. If you can find a lightweight library that does that, maybe we could consider setting an option on the RST parser to transform docstrings into Markdown. Well, I'll let you share your thoughts!

@plannigan
Copy link
Contributor Author

That is a helpful intro into the parser API. I'll play around with it this weekend and see what I come up with.

That is a good point about RST, it is a large specification. For the initial version, I was thinking about parsing out the fields I linked to in the initial request (which seems to match what is supported by the google format). This would focus on the documentation of a functions contract, without trying to cover the whole RST specification. This would be sufficient for my needs. If there is a future request for more full support, I could look into using something like docutils to do the parsing and accessing the data it produces.

@plannigan
Copy link
Contributor Author

Status update: Things are coming along nicely. I haven't run into any hard blockers. Once I get things into a good state, I'll try to break it into smaller PRs so it isn't big dump of >1k lines.

Other Notes:

  • I've found a few fixtures in the test suite that are using the Google format. Not a huge deal as I've been able to make equivalent fixtures. Going forward, it would probably make sense to move those into a dedicated directory.
  • I've got some ideas that I think would improve the structure. This mostly has to deal with using dataclasses instead of dictionaries. For example, the context dictionary is very flexible, but it makes it hard to know what keys are expected to exist, what keys are optional and there is a lot of type information that gets lost when it gets passed from one place to another.
    For now I have a ParseContext class that I am using internally in my implementation. It seems to be working well, but it is hard to know how much that would effect the rest of the code as it is hard to know what else is being placed in that dictionary. If this is a non-starter, I can back it out and use the dictionary within my implementation.

@pawamoy
Copy link
Member

pawamoy commented Oct 17, 2020

Great!

About the fixtures: sure, we should move them where relevant, and make sure to remove unnecessary sections.

I've got some ideas that I think would improve the structure.

Yes! Music to my ears 😄

You are totally right about the context dictionary, I don't like it either. Honestly I think pytkdocs needs a whole refactor, but that's another story (see #64).

Once I get things into a good state, I'll try to break it into smaller PRs so it isn't big dump of >1k lines.

Sounds good to me! Thank you for your contributions!

plannigan added a commit to plannigan/pytkdocs that referenced this issue Oct 18, 2020
This implementation should be mostly complete.

Not implemented:
 - references: Not planned at this time
 - examples: I'm not aware of a standard format for this

Still needed:
 - more test:
   - Many of the error cases aren't currently covered
   - More full fixture test cases

To be cleaned up:
 - I'm not confident I got the types right for all of the data that comes in with the context
 - ParseContext works, but it is a bit awkward with the rest of the project using the dict

References: mkdocstrings#67
pawamoy pushed a commit that referenced this issue Nov 28, 2020
This implementation should be mostly complete.

Not implemented:
 - references: Not planned at this time
 - examples: I'm not aware of a standard format for this

Issue #67: #67
PR #71: #71
@pawamoy
Copy link
Member

pawamoy commented Nov 28, 2020

Initial support in #71

@pawamoy pawamoy closed this as completed Nov 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants