multilined cpp function signatures split between param name and datatype

[2bndy5]

I'm getting ready to start exporting docygen XML to sphinx-immaterial (via breathe ext), and I noticed that function signatures that are wrapped to multiple lines are confusingly split between the parameter's datatype and name. Observe a test I made without doxygen/breathe (using standard sphinx cpp domain directives):

.. cpp:function:: const MyType Foo(const MyType bar, uint8_t baz, bool flag, uint16_t foobar, int32_t foobaz, unsigned long barbaz)

   Some function description.

I'm suspecting the fix should be placed in apidoc_formatting.py of this theme, but I still need to research more about how it detects the directive's domain.

[jbms]

The parameter wrapping is implemented here:

sphinx-immaterial/sphinx_immaterial/apidoc_formatting.py

Line 148 in ec14e7c

def visit_desc_signature(self, node: sphinx.addnodes.desc_signature) -> None:

sphinx-immaterial/src/assets/stylesheets/main/_api.scss

Line 68 in ec14e7c

&.sig-wrap {

It looks like there will indeed need to be some tweaking of the C++ api formatting, in several different ways.

Arguably the wrapping (for all languages) should be done using a code formatter rather than CSS like I have done, since you may need several levels of nesting for complicated template parameter lists, etc., but that might be a bit tricky.

It looks like the desc_signature node includes the domain (e.g. "cpp") in its list of "classes", so you could potentially use that to detect it.

[2bndy5]

I kinda like the css approach. I believe that would allow for custom member icons in the secondary ToC

[jbms]

Arguably the bug here is in the sphinx cpp domain itself --- the desc_parameter node should contain the type as well, but currently it only contains the parameter name.

As far as adding icons to the secondary TOC for C++ "objects", I think that is independent of this issue, and could be done by adding some entries here:

sphinx-immaterial/sphinx_immaterial/nav_adapt.py

Line 203 in ec14e7c

OBJECT_ICON_INFO: Dict[Tuple[str, str], ObjectIconInfo] = {

[2bndy5]

At first glance, I kinda figured this issue wasn't exclusive to this theme. But, I'm glad I asked here first before taking this upstream. I still have to fully understand it all before taking any action whatsoever. Thanks again for the invaluable leads.

[jbms]

While the structure of the desc_parameter nodes may not be "correct" for the cpp domain, it may be that other themes do not make use of the structure like this theme does, and so it does not matter for them.

[2bndy5]

Yeah I couldn't (readily) find a theme that separates the signature into multiple lines. Everything I've seen so far just shows the signature in 1 line (length be damned).

[2bndy5]

I was able to solve this in CSS, but it is ugly as hell. I'll keep learning docutils (whose docs are ironically terrible), but for now I think I'll just add custom CSS to my projects until a more robust solution can be realized.

.md-typeset dl.api-field>dt.sig-wrap .sig-param + .sig-param::before,
.md-typeset dl.objdesc>dt.sig-wrap .sig-param + .sig-param::before,
.md-typeset dl.api-field>dt.sig-wrap .n.sig-param + .kt::before,
.md-typeset dl.objdesc>dt.sig-wrap .n.sig-param + .kt::before,
.md-typeset dl.api-field>dt.sig-wrap .n.sig-param + .n:not(.sig-param)::before,
.md-typeset dl.objdesc>dt.sig-wrap .n.sig-param + .n:not(.sig-param)::before {
    content: "\a     ";
    white-space: break-spaces;
}

This still doesn't handle template parameters and the first arg to long function signatures... The later wouldn't be hard, but template descriptions is bit more complex since the entire signature is parsed from 1 long line - implying that we'd have to parse more than 1 set of brackets and determine if template/function/both parameters needs multiple lines.

[2bndy5]

I've started expanding on the CSS solution currently implemented. I also went through and made sure that syntax highlighting is triggered properly (while still maintaining the a.reference override within signatures).

python

c++

---

I did look into patching the sphinx domains, but they are wildly different in construction. Also, I think the CSS approach can cover other added domains (not builtin to sphinx).

[jbms]

I have actually been working heavily on generating C++ API documentation for TensorStore, and in the process have accumulated a large number of fixes to make the C++ domain work better for this theme (and work better in general). (Unfortunately they aren't really polished yet, as they are still a work in progress.)

Initially I attempted to implement some wrapping rules for template parameters, function parameters, etc. However, C++ syntax is quite complicated, and at least for my use cases many different parts of the declarations might require wrapping, so I kept having to add additional wrapping rules.

Therefore I abandoned that approach and instead implemented a sphinx extension that uses clang-format to format the C++ signatures. Using an existing formatter is probably also the way to go for Python and other languages.

The advantage of clang-format is that it already has quite elaborate wrapping rules and provides a lot of customization via the style options.

There is the PyPI clang-format package so there is no need for users to install it separately.

Here is the code:

https://gist.github.com/jbms/90e88d2c7a539b73614005bca02531bc

The basic challenges are:

1. We don't want to start a separate clang-format for every single signature, so we collect all of the signatures and then run clang-format once.

2. clang-format operates on text, but we need to preserve the docutils node structure. Therefore we convert the signature to text, format it, then walk the docutils nodes to match up the text to the clang-formatted text, and adjust the whitespace.

The same code could also be adapted to support other formatters/other languages pretty easily.

[2bndy5]

ok! I'll separate the signature changes from the the other changes in the branch I'm working on.

I have some experience with clang-format (& clang-tidy).

• clang-format is very rigid in style format (its rather hard to specify a custom style that suites one's needs.

• It is very important what version of clang-format is used. The style options change and some settings are not available in newer versions. Thus v13 is not completely compatible with v12

  I think v10 is still the default installed via apt (LTS releases are right around the corner), but the LLVM project is up to v14 now.

app.add_config_value(name='cpp_signature_clang_format_style', default='', rebuild='env')

It would be clearer to users that the default style used in clang-format is the LLVM style. Users could change this to file if there is a .clang-format config file (yaml syntax) present (in the docs/parent folder??). Personally, I favor the somewhat relaxed Webkit style.

[jbms]

clang-format has quite a lot of options (https://clang.llvm.org/docs/ClangFormatStyleOptions.html) --- but I suppose there can always be more.

I would suggest depending on the PyPI package rather than relying on a system-installed binary --- that way the version is consistent. The PyPI package has version 13.0.1. Unfortunately it does not have a later version though maybe they could be prodded into making one.

I intended that users would indeed specify a custom style using that option. I think in many cases you may want a different style in the documentation from what is used in the source code. In particular for this theme the ColumnLimit should be set to 70. But yeah, perhaps file is a reasonable default.

[2bndy5]

Is is ok if I just augment my revised CSS solution with :not(.cpp)? I feel the CSS is robust enough for most languages.

Could we use clang-format extension for any language that clang-format supports? Or is it designed only for C++?

[jbms]

If you think your CSS solution is preferable to using an existing formatter in some cases then it seems reasonable for us to have that as well, for users that don't want to rely on an external formatter. It just seemed like to handle all of the cases you would end up having to basically implement a complete formatter, which is not something I wanted to do --- better to reuse the existing work. I think it doesn't necessarily need to explicitly exclude cpp --- we can probably arrange that a different way, e.g. via the sig-wrap class.

The clang-format approach definitely could be used for any language supported by clang-format --- and in fact there is very little specific to clang-format --- it could be made more generic to work with basically any formatter for any language.

[2bndy5]

It just seemed like to handle all of the cases you would end up having to basically implement a complete formatter, which is not something I wanted to do --- better to reuse the existing work

I'm totally behind this idea. I'm just looking to fool-proof the CSS.

via the sig-wrap class.

Yeah, that's what I meant. Unless I misunderstanding that comment.

.sig-wrap:not(.cpp) {
  // ...
}

[jbms]

I think we can just arrange that sig-wrap is not set on signatures when the external formatter has been used --- so there is no need to additionally exclude cpp except if your CSS doesn't work for C++.

[2bndy5]

Well It definitely works for C++ function args (doesn't account for template args). Ok, I'll not exclude cpp class in CSS.

[2bndy5]

Any preference to handling the optional delimiter in arg lists?

My first instinct is to indent an additional level, but that may be rather difficult with only CSS.

[jbms]

Personally I don't really care for that syntax at all; it isn't real Python syntax, which means it can't be parsed into an AST properly, and therefore can't be properly cross-linked.

Python syntax already provides a way to indicate optional arguments, so I don't see the point of that alternate syntax.

[2bndy5]

I totally agree. Should I remove it? The docs that are referenced as the original RST source no longer uses the optional syntax.

[jbms]

Yeah let's remove it.

[2bndy5]

I found another sample that uses the optional syntax, but I'm leaving it in as it doesn't require multiple lines.

ps - I've implemented your idea of a directive that reduces example snippet duplication. This screenshot is a bit of a tease.