Paragraphs in block doc comments are broken

[Cldfire]

I returned to a crate of mine for the first time in quite a while and was perplexed to see that all of the paragraphs in my doc comments were gone, and instead had been turned into line breaks (so all of the paragraphs were squashed together).

Upon investigation I discovered that this was due to my usage of block doc comments (/** */) rather than the more common /// style. It occurs on both the current stable and nightly release.

Source comment:

    /**
    Gets the checksum of the config stored in this `Device`'s infoROM.
    
    Can be used to make sure that two GPUs have the exact same configuration.
    The current checksum takes into account configuration stored in PWR and ECC
    infoROM objects. The checksum can change between driver released or when the
    user changes the configuration (e.g. disabling/enabling ECC).
    
    # Errors

    * `CorruptedInfoROM`, if this `Device`'s checksum couldn't be retrieved due to infoROM corruption
    * `Uninitialized`, if the library has not been successfully initialized
    * `NotSupported`, if this `Device` does not support this feature
    * `GpuLost`, if this `Device` has fallen off the bus or is otherwise inaccessible
    * `Unknown`, on any unexpected error
    
    # Device Support

    Supports all devices with an infoROM.
    */

Output when using block doc comments:

And the HTML:

Output after deleting the block doc comment start / end syntax and placing /// in front of each line (this is the expected output for both):

And the HTML for that:

[Centril]

@Cldfire Could you try bisecting the regression to an old Rust version to see when it about happened?

[Cldfire]

Yeah, I plan on investigating this and putting a PR together today.

[Cldfire]

Update: I've pinpointed the cause of the issue to #60140 (commit 3030164). The broken behavior was introduced by the upgrade to pulldown-cmark 0.4.1.

I looked over the commit and nothing stood out to me as being the potential cause, which implies that the issue is within pulldown-cmark itself. In any case, I don't really have the time to go digging deeper into the library at the moment; perhaps someone with knowledge in this area has an idea of what specifically could be responsible for this?

[Cldfire]

I investigated a little more and found out that this issue appears only when I have whitespace in the blank lines between paragraphs (something that my editor inserts due to how it handles indentation).

Let's work from this example.

/**
This is a doc comment!

It should have blank lines.

Between these paragraphs.
*/
pub fn a_function() {

}

Documenting this results in:

Correct!

Indenting all lines of the comment (like you would to align the comment with the indent of a method in an impl block, for instance):

    /**
    This is a doc comment!

    It should have blank lines.

    Between these paragraphs.
    */
pub fn a_function() {

}

Doc output:

Correct again!

However, if I add indentation in the blank lines (screenshot from editor):

The output is broken:

The same exact behavior happens with /// comments, actually. Inserting > 3 spaces in a blank line between a paragraph replaces the blank line with a line break. This also does not occur on 1.35.0 (before the pulldown-cmark upgrade). On 1.35.0 I can insert as much whitespace as I want between paragraphs and it doesn't turn into a line break.

So I guess this is less "paragraphs are broken in block doc comments" and more "the way pulldown-cmark handles whitespace changed." The question now is, is the new behavior correct? If so this issue can be closed and I will adjust my crate accordingly (aka remove all usages of block doc comments, because this particular behavior would make them far more of a hassle to use than what they're worth).

[ehuss]

is the new behavior correct?

I do not think so. My reading of the commonmark spec is that a "blank line" may contain spaces. I have filed pulldown-cmark/pulldown-cmark#370 for this issue.

[Cldfire]

Cool, thanks!