Documentation

[Cloudef]

It would be nice to have documentation syntax in code so we can generate for example man pages for wlc API. Doxygen has been used before, but it may be worth to resee what are the options nowadays.

[Earnestly]

Here is a tentative (i.e. braindump) mockup that demonstrates what this could look like.

Given the function wlc_init, the comment block above it would look something like this:

/* -- NAME
 * initialize the wayland compositor
 *
 * -- DESCRIPTION
 * wlc_init() initializes the `interface` along with passing `argc` and
 * `argv` from main(), so wlc can rename the process it forks to cleanup
 * crashed parent process and do FD passing (non-logind).
 *
 * -- RETURN VALUE
 * Upon successful execution wlc_init() will return true.
 *
 * -- ERRORS
 * All errors encountered in wlc_init() are considered fatal and thus should
 * never really return false.
 *
 * -- NOTES
 * Avoid running unverified code before wlc_init() as wlc compositor may be run
 * with higher privileges on non logind systems where compositor binary needs
 * to be suid.
 *
 * -- ALSO SEE
 * wlc_interface(3)
 */
WLC_NONULLV(1) bool wlc_init(const struct wlc_interface *interface, int argc, char *argv[]);

The converstion to groff_man(7) should yeild something like this:

The backticks are expanded to .I (italics) or \fI...\fR.

Any reference to a function name in the form of foobar() is expanded out to .BR foobar () or \fBfoobar\fR().

Note that the entire SYNOPSIS section should be generated, the point of this format is to be both flexible, very man-pages(7) specific in layout and try to not let the generation get in the way of the author.

.TH WLC_INIT 3 2015-09-30 WLC "WLC API Functions"

.SH NAME
wlc_init \- initialize the wayland compositor

.SH SYNOPSIS
.B #include <wlc/wlc.h>

.BI "bool wlc_init(const struct wlc_interface " *interface ", int " argc ", char " *argv[] ");"

.SH DESCRIPTION
.BR wlc_init ()
initializes the
.I interface
along with passing
.I argc
and
.I argv
from
.BR main ()
, so wlc can rename the process it forks to cleanup crashed parent process and do FD passing (non-logind).

.SH RETURN VALUE
Upon successful execution
.BR wlc_init ()
will return true.

.SH ERRORS
All errors encountered in
.BR wlc_init ()
are considered fatal and thus should never really return false.

.SH NOTES
Avoid running unverified code before
.BR wlc_init ()
as wlc compositor may be run with higher privileges on non logind systems where compositor binary needs to be suid.

.SH ALSO SEE
.BR wlc_interface (3)

Which would should render something like this on the users screen via the normal man command:

WLC_INIT(3)                 WLC API Functions                WLC_INIT(3)

NAME
       wlc_init - initialise the wayland compositor

SYNOPSIS
       #include <wlc/wlc.h>

       bool  wlc_init(const  struct  wlc_interface *interface, int argc,
       char *argv[]);

DESCRIPTION
       wlc_init() initializes the interface along with passing argc  and
       argv  from  main()  ,  so  wlc can rename the process it forks to
       cleanup crashed parent process and do FD passing (non-logind).

RETURN VALUE
       Upon successful execution wlc_init() will return true.

ERRORS
       All errors encountered in wlc_init()  are  considered  fatal  and
       thus should never really return false.

NOTES
       Avoid running unverified code before wlc_init() as wlc compositor
       may be run with higher privileges on  non  logind  systems  where
       compositor binary needs to be suid.

ALSO SEE
       wlc_interface(3)

WLC                            2015-09-30                    WLC_INIT(3)

[Cloudef]

This is not bad at all, and doesn't look too complicated to write generator for (apart from understanding bit of C for the function prototypes). If escaping is desired, it should be in way that is simplest for the parser. That is, do not provide many ways to escape things.

[vodik]

I wonder if clang can do it

[Cloudef]

Clang was bit troubling with python at least. I've never used the C api though. Ragel can parse C nicely, but won't "understand" it. I think most documentation generators don't actually understand C, but I can see that becoming problematic in some scenarios, like having preprocessor definitions (often for symbol visibility, or compiler attributes) front of function declarations, or even worse libpng style macro function declarations.

[Earnestly]

There are several limitations to this approach. The main one is you're essentially locking yourself to groff_man(7) as the bugs section of man(7) points out:

Most  of  the  macros  describe  formatting (e.g., font type and spacing)
instead of marking semantic content (e.g., this text is  a  reference  to
another  page),  compared to formats like mdoc and DocBook (even HTML has
more semantic markings).  This situation makes it harder to vary the  man
format for different media, to make the formatting consistent for a given
media, and to automatically insert cross-references.  By sticking to  the
safe  subset described above, it should be easier to automate transition‐
ing to a different reference page format in the future.

When doing parsing of semantic elements like the backticks one has to consider how it could be represented.

For example consider the text:

`foo bar()`

If we define backticks as .I and bar() as .BR bar () then the non-inline solution would be:

.I foo
.BI bar ()

This immediately presents lots of edge cases which would be ugly to parse. The first is that bar() can no longer simply transform to .BR bar (), you need to know that you're inside italics mode. This gets worse when you consider trailing punctution:

`foo bar(), baz`

Would become the following groff without inlining the macros:

.I foo
.BR bar () "" ,
.I baz

The use of "" above is to satify the nature of .BR which alternates each word with bold and roman (normal text), the "" is there to absorb the next bold effect.

However there is a rather nice solution to both of these issues but generally produces slightly less than pretty looking manuals. This is the use of inline macros via the \f escape.

Consider again the first example of bold being used inside italics. The inline solution (from a parsers perspective) to this would be:

\fIfoo \fBbar\fR\fI()\fR

And for the second example:

\fIfoo \Bbar\fR\fI(), bar\fR

Notice how we can allow unbalanced pairs, the first \fI does not have a corresponding \fR as it is consumed by the \fR which terminates the \fB which preceeded it, \fB itself also consuming the effect of \fI. This is quite nice for parsers as they wouldn't need to care, each element can be treated in isolated.

That is to say, bar() can be translated to \fBbar\fR() directly without needing to worry about the surrounding context. Likewise, a pair of backticks can be replaced with \fI and \fR respectively regardless of the contect.

Things to consider would be if you want this format to be more general. This would almost certainly require the use of an IR, which the prompts the question of why not just use a well supported markup like restructed text or asciidoc directly? These formats can both generate manuals fairly comfortably, all the tool here would need to do is parse the comment block and C definition below it.

If the manual generation isn't perfectly like man-pages(7) then it probably wouldn't be too difficult to adjust the output too.

[Earnestly]

This is quite nice too https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt

[Earnestly]

Heh, regex poorly all the things: (but as proof of concept it seems to work for the simple stuff I've thrown at, including some edge cases.)

# The .TH and SYNOPSIS sections would be generated from elsewhere.
teapot earnest %i master ~ cat foo                       
.TH WLC_INIT 3 2015-09-30 WLC "WLC API Functions"

-- NAME
initialize the wayland compositor

.SH SYNOPSIS
.B #include <wlc/wlc.h>

.BI "bool wlc_init(const struct wlc_interface " *interface ", int " argc ", char " *argv[] ");"

-- DESCRIPTION
wlc_init() initializes the `interface` along with passing `argc` and
`argv` from main(), so wlc can rename the process it forks to cleanup
crashed parent process and do FD passing (non-logind).

-- RETURN VALUE
Upon successful execution wlc_init() will return true.

-- ERRORS
All errors encountered in wlc_init() are considered fatal and thus should
never really return false.

-- NOTES
Avoid running unverified code before wlc_init() as wlc compositor may be run
with higher privileges on non logind systems where compositor binary needs
to be suid.

-- ALSO SEE
wlc_interface(3), wlc_output(3)

teapot earnest %i master ~ gman < foo
.TH WLC_INIT 3 2015-09-30 WLC "WLC API Functions"

.SH NAME
initialize the wayland compositor

.SH SYNOPSIS
.B #include <wlc/wlc.h>

.BI "bool wlc_init(const struct wlc_interface " *interface ", int " argc ", char " *argv[] ");"

.SH DESCRIPTION
\fBwlc_init\fR() initializes the \fIinterface\fR along with passing \fIargc\fR and
\fIargv\fR from \fBmain\fR(), so wlc can rename the process it forks to cleanup
crashed parent process and do FD passing (non-logind).

.SH RETURN VALUE
Upon successful execution \fBwlc_init\fR() will return true.

.SH ERRORS
All errors encountered in \fBwlc_init\fR() are considered fatal and thus should
never really return false.

.SH NOTES
Avoid running unverified code before \fBwlc_init\fR() as wlc compositor may be run
with higher privileges on non logind systems where compositor binary needs
to be suid.

.SH ALSO SEE
\fBwlc_interface\fR(3), \fBwlc_output\fR(3)

gman() { italic | embold | mkref | mksec }

italic() { sed 's/`\([^`\]*\)`/\\fI\1\\fR/g'; }
embold() { sed 's/\b\([^ ]*\)()/\\fB\1\\fR()/g'; }
mkref() { sed 's/\b\([^ ]*\)(\([0-9]\))/\\fB\1\\fR(\2)/g'; }
mksec() { sed 's/^-- \(.*\)/.SH \1/'; }

[Cloudef]

That's nice proof of concept.

Things to consider would be if you want this format to be more general. This would almost certainly require the use of an IR, which the prompts the question of why not just use a well supported markup like restructed text or asciidoc directly? These formats can both generate manuals fairly comfortably, all the tool here would need to do is parse the comment block and C definition below it.

This is fine too, as long as the output is high quality. The only downside might be that the comment blocks might become more markup heavy. I don't think most documentation derives too much from what you have done with wlc_init here. Though I can't consider other projects here.