Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: mkdocstrings/pytkdocs
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 0.5.2
Choose a base ref
...
head repository: mkdocstrings/pytkdocs
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 0.6.0
Choose a head ref
  • 4 commits
  • 8 files changed
  • 1 contributor

Commits on Jun 14, 2020

  1. Copy the full SHA
    02c0042 View commit details
  2. Copy the full SHA
    4d21ea1 View commit details
  3. docs: Update readme

    pawamoy committed Jun 14, 2020
    Copy the full SHA
    b4606fd View commit details
  4. chore: Prepare release 0.6.0

    pawamoy committed Jun 14, 2020
    Copy the full SHA
    1f89024 View commit details
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -5,6 +5,14 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

<!-- insertion marker -->
## [v0.6.0](https://github.com/pawamoy/pytkdocs/releases/tag/v0.6.0) - 2020-06-14

<small>[Compare with v0.5.2](https://github.com/pawamoy/pytkdocs/compare/v0.5.2...v0.6.0)</small>

### Features
- Support attributes sections for Google-style docstrings ([02c0042](https://github.com/pawamoy/pytkdocs/commit/02c0042f9d4d8ab799550418d8474d1a6669feec) by Timothée Mazzucotelli).


## [v0.5.2](https://github.com/pawamoy/pytkdocs/releases/tag/v0.5.2) - 2020-06-11

<small>[Compare with v0.5.1](https://github.com/pawamoy/pytkdocs/compare/v0.5.1...v0.5.2)</small>
109 changes: 107 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -54,16 +54,112 @@ Input format:
{
"objects": [
{
"path": "my_module.my_class",
"path": "pytkdocs",
"members": true,
"inherited_members": false,
"filters": [
"!^_[^_]"
]
],
"docstring_style": "google",
"docstring_options": {
"replace_admonitions": true
}
}
]
}
```

Output format:

```json
{
"loading_errors": [
"string (message)"
],
"parsing_errors": {
"string (object)": [
"string (message)"
]
},
"objects": [
{
"name": "pytkdocs",
"path": "pytkdocs",
"category": "module",
"file_path": "/media/data/dev/pawamoy/pytkdocs/src/pytkdocs/__init__.py",
"relative_file_path": "pytkdocs/__init__.py",
"properties": [
"special"
],
"parent_path": "pytkdocs",
"has_contents": true,
"docstring": "pytkdocs package.\n\nLoad Python objects documentation.",
"docstring_sections": [
{
"type": "markdown",
"value": "pytkdocs package.\n\nLoad Python objects documentation."
}
],
"source": {
"code": "\"\"\"\npytkdocs package.\n\nLoad Python objects documentation.\n\"\"\"\n\nfrom typing import List\n\n__all__: List[str] = []\n",
"line_start": 1
},
"children": {
"pytkdocs.__all__": {
"name": "__all__",
"path": "pytkdocs.__all__",
"category": "attribute",
"file_path": "/media/data/dev/pawamoy/pytkdocs/src/pytkdocs/__init__.py",
"relative_file_path": "pytkdocs/__init__.py",
"properties": [
"special"
],
"parent_path": "pytkdocs",
"has_contents": false,
"docstring": null,
"docstring_sections": [],
"source": {},
"children": {},
"attributes": [],
"methods": [],
"functions": [],
"modules": [],
"classes": []
}
},
"attributes": [
"pytkdocs.__all__"
],
"methods": [],
"functions": [],
"modules": [
"pytkdocs.__main__",
"pytkdocs.cli",
"pytkdocs.loader",
"pytkdocs.objects",
"pytkdocs.parsers",
"pytkdocs.properties",
"pytkdocs.serializer"
],
"classes": []
}
]
}
```

## Command-line

Running `pytkdocs` without argument will read the whole standard input,
and output the result once.

Running `pytkdocs --line-by-line` will enter an infinite loop,
where at each iteration one line is read on the standard input,
and the result is written back on one line.
This allows other programs to use `pytkdocs` in a subprocess,
feeding it single lines of JSON, and reading back single lines of JSON as well.
This mode was actually implemented specifically for
[mkdocstrings](https://github.com/pawamoy/mkdocstrings).

## Configuration

The configuration options available are:
@@ -83,3 +179,12 @@ The configuration options available are:
- `members`: this option allows to explicitly select the members of the top-object.
If `True`, select every members that passes filters. If `False`, select nothing.
If it's a list of names, select only those members, and apply filters on their children only.

- `docstring_style`: the docstring style to use when parsing the docstring. Only one parser available: `google`.

- `docstring_options`: options to pass to the docstring parser.
- `google` accepts a `replace_admonitions` boolean option (default: true). When enabled, this option will
replace titles of an indented block by their Markdown admonition equivalent:
`AdmonitionType: Title` will become `!!! admonitiontype "Title"`.

- `inherited_members`: true or false (default). When enabled, inherited members will be selected as well.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ build-backend = "poetry.masonry.api"

[tool.poetry]
name = "pytkdocs"
version = "0.5.2"
version = "0.6.0"
description = "Load Python objects documentation."
authors = ["Timothée Mazzucotelli <pawamoy@pm.me>"]
license = "ISC License"
19 changes: 18 additions & 1 deletion src/pytkdocs/parsers/docstrings/base.py
Original file line number Diff line number Diff line change
@@ -22,6 +22,22 @@ def __init__(self, annotation: Any, description: str) -> None:
self.description = description


class Attribute(AnnotatedObject):
"""A helper class to store information about a documented attribute."""

def __init__(self, name: str, annotation: Any, description: str) -> None:
"""
Initialization method.
Arguments:
name: The attribute's name.
annotation: The object's annotation.
description: The object's description.
"""
super().__init__(annotation, description)
self.name = name


class Parameter(AnnotatedObject):
"""A helper class to store information about a signature parameter."""

@@ -90,6 +106,7 @@ class Type:
EXCEPTIONS = "exceptions"
RETURN = "return"
EXAMPLES = "examples"
ATTRIBUTES = "attributes"

def __init__(self, section_type: str, value: Any) -> None:
"""
@@ -125,7 +142,7 @@ def __init__(self) -> None:
self.context: dict = {}
self.errors: List[str] = []

def parse(self, docstring: str, context: Optional[dict] = None,) -> Tuple[List[Section], List[str]]:
def parse(self, docstring: str, context: Optional[dict] = None) -> Tuple[List[Section], List[str]]:
"""
Parse a docstring and return a list of sections and parsing errors.
116 changes: 72 additions & 44 deletions src/pytkdocs/parsers/docstrings/google.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,25 @@
"""This module defines functions and classes to parse docstrings into structured data."""
import re
from typing import Any, List, Optional, Pattern, Sequence, Tuple

from pytkdocs.parsers.docstrings.base import AnnotatedObject, Parameter, Parser, Section, empty

TITLES_PARAMETERS: Sequence[str] = ("args:", "arguments:", "params:", "parameters:")
"""Titles to match for "parameters" sections."""

TITLES_EXCEPTIONS: Sequence[str] = ("raise:", "raises:", "except:", "exceptions:")
"""Titles to match for "exceptions" sections."""

TITLES_RETURN: Sequence[str] = ("return:", "returns:")
"""Titles to match for "returns" sections."""

TITLES_EXAMPLES: Sequence[str] = ("example:", "examples:")
"""Titles to match for "examples" sections."""
from typing import Any, List, Optional, Pattern, Tuple

from pytkdocs.parsers.docstrings.base import AnnotatedObject, Attribute, Parameter, Parser, Section, empty

SECTIONS_TITLES = {
"args:": Section.Type.PARAMETERS,
"arguments:": Section.Type.PARAMETERS,
"params:": Section.Type.PARAMETERS,
"parameters:": Section.Type.PARAMETERS,
"raise:": Section.Type.EXCEPTIONS,
"raises:": Section.Type.EXCEPTIONS,
"except:": Section.Type.EXCEPTIONS,
"exceptions:": Section.Type.EXCEPTIONS,
"return:": Section.Type.RETURN,
"returns:": Section.Type.RETURN,
"example:": Section.Type.EXAMPLES,
"examples:": Section.Type.EXAMPLES,
"attribute:": Section.Type.ATTRIBUTES,
"attributes:": Section.Type.ATTRIBUTES,
}

RE_GOOGLE_STYLE_ADMONITION: Pattern = re.compile(r"^(?P<indent>\s*)(?P<type>[\w-]+):((?:\s+)(?P<title>.+))?$")
"""Regular expressions to match lines starting admonitions, of the form `TYPE: [TITLE]`."""
@@ -32,12 +37,21 @@ def __init__(self, replace_admonitions: bool = True) -> None:
"""
super().__init__()
self.replace_admonitions = replace_admonitions
self.section_reader = {
Section.Type.PARAMETERS: self.read_parameters_section,
Section.Type.EXCEPTIONS: self.read_exceptions_section,
Section.Type.EXAMPLES: self.read_examples_section,
Section.Type.ATTRIBUTES: self.read_attributes_section,
Section.Type.RETURN: self.read_return_section,
}

def parse_sections(self, docstring: str) -> List[Section]: # noqa: D102
if "signature" not in self.context:
self.context["signature"] = getattr(self.context["obj"], "signature", None)
if "annotation" not in self.context:
self.context["annotation"] = getattr(self.context["obj"], "type", empty)
if "attributes" not in self.context:
self.context["attributes"] = {}

sections = []
current_section = []
@@ -55,39 +69,13 @@ def parse_sections(self, docstring: str) -> List[Section]: # noqa: D102
in_code_block = False
current_section.append(lines[i])

elif line_lower in TITLES_PARAMETERS:
if current_section:
if any(current_section):
sections.append(Section(Section.Type.MARKDOWN, "\n".join(current_section)))
current_section = []
section, i = self.read_parameters_section(lines, i + 1)
if section:
sections.append(section)

elif line_lower in TITLES_EXCEPTIONS:
if current_section:
if any(current_section):
sections.append(Section(Section.Type.MARKDOWN, "\n".join(current_section)))
current_section = []
section, i = self.read_exceptions_section(lines, i + 1)
if section:
sections.append(section)

elif line_lower in TITLES_RETURN:
if current_section:
if any(current_section):
sections.append(Section(Section.Type.MARKDOWN, "\n".join(current_section)))
current_section = []
section, i = self.read_return_section(lines, i + 1)
if section:
sections.append(section)

elif line_lower in TITLES_EXAMPLES:
elif line_lower in SECTIONS_TITLES:
if current_section:
if any(current_section):
sections.append(Section(Section.Type.MARKDOWN, "\n".join(current_section)))
current_section = []
section, i = self.read_examples_section(lines, i + 1)
section_reader = self.section_reader[SECTIONS_TITLES[line_lower]]
section, i = section_reader(lines, i + 1)
if section:
sections.append(section)

@@ -296,6 +284,46 @@ def read_parameters_section(self, lines: List[str], start_index: int) -> Tuple[O
self.error(f"Empty parameters section at line {start_index}")
return None, i

def read_attributes_section(self, lines: List[str], start_index: int) -> Tuple[Optional[Section], int]:
"""
Parse an "attributes" section.
Arguments:
lines: The parameters block lines.
start_index: The line number to start at.
Returns:
A tuple containing a `Section` (or `None`) and the index at which to continue parsing.
"""
attributes = []
block, i = self.read_block_items(lines, start_index)

for attr_line in block:
try:
name_with_type, description = attr_line.split(":", 1)
except ValueError:
self.error(f"Failed to get 'name: description' pair from '{attr_line}'")
continue

description = description.lstrip()

if " " in name_with_type:
name, annotation = name_with_type.split(" ", 1)
annotation = annotation.strip("()")
if annotation.endswith(", optional"):
annotation = annotation[:-10]
else:
name = name_with_type
annotation = self.context["attributes"].get(name, {}).get("annotation", empty)

attributes.append(Attribute(name=name, annotation=annotation, description=description))

if attributes:
return Section(Section.Type.ATTRIBUTES, attributes), i

self.error(f"Empty attributes section at line {start_index}")
return None, i

def read_exceptions_section(self, lines: List[str], start_index: int) -> Tuple[Optional[Section], int]:
"""
Parse an "exceptions" section.
21 changes: 20 additions & 1 deletion src/pytkdocs/serializer.py
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@
from typing import Any, Optional, Pattern

from pytkdocs.objects import Object, Source
from pytkdocs.parsers.docstrings.base import AnnotatedObject, Parameter, Section
from pytkdocs.parsers.docstrings.base import AnnotatedObject, Attribute, Parameter, Section

try:
from typing import GenericMeta # python 3.6
@@ -84,6 +84,23 @@ def serialize_annotated_object(obj: AnnotatedObject) -> dict:
return {"description": obj.description, "annotation": annotation_to_string(obj.annotation)}


def serialize_attribute(attribute: Attribute) -> dict:
"""
Serialize an instance of [`Attribute`][pytkdocs.parsers.docstrings.base.Attribute].
Arguments:
attribute: The attribute to serialize.
Returns:
A JSON-serializable dictionary.
"""
return {
"name": attribute.name,
"description": attribute.description,
"annotation": annotation_to_string(attribute.annotation),
}


def serialize_parameter(parameter: Parameter) -> dict:
"""
Serialize an instance of [`Parameter`][pytkdocs.parsers.docstrings.base.Parameter].
@@ -166,6 +183,8 @@ def serialize_docstring_section(section: Section) -> dict:
serialized.update({"value": [serialize_annotated_object(e) for e in section.value]}) # type: ignore
elif section.type == section.Type.PARAMETERS:
serialized.update({"value": [serialize_parameter(p) for p in section.value]}) # type: ignore
elif section.type == section.Type.ATTRIBUTES:
serialized.update({"value": [serialize_attribute(p) for p in section.value]}) # type: ignore
elif section.type == section.Type.EXAMPLES:
serialized.update({"value": section.value}) # type: ignore
return serialized
Loading