Rustdoc 2019 roadmap planning

[QuietMisdreavus]

As we get close to the end of the year, it would be helpful to try to organize a direction to push rustdoc next year. So far we've been fairly free-form about what we work on, but if we have a few overarching goals to point at, it would be easier to try to attract new contributors or to prioritize work.

That said, what does the team want to see from rustdoc in the coming year? Feel free to write up ideas in this thread. I'll start with some comments of my own that are outstanding items or are things that i can think of.

cc @steveklabnik @GuillaumeGomez @ollie27 @onur (since i can't ping rust-lang teams all at once on here 😅)

[QuietMisdreavus]

Currently unstable major features

Rustdoc has a handful of features that require nightly, either because they were introduced recently or their implementation is incomplete. Maybe we can try to push stabilization on some of these? I'm going to mention items i'd personally like to see stabilized before the end of next year, but there are probably other smaller ones we could add much more quickly.

• #[doc(include="...")]
  • As far as i know, this just needs someone to push for it? I haven't looked at it in a while.
• #[doc(cfg(...))]
  • If i remember right, there is an issue with platform-specific items being returned from functions, but otherwise, the core feature works fine?
• Intra-doc links
  • This probably needs the most work. There are still some spurious resolution errors being emitted, and we're still missing proper cross-crate resolution. We'll likely need to coordinate with someone from T-compiler to push this over the line.

[QuietMisdreavus]

Testing

There's an open PR right now that aims to add a new form of test for rustdoc: visual regression testing. This is meant to avert situations like rust-lang/rust#53262 where we lost type-kind colors in certain situations, but it can also detect things like items moving around on the page without us meaning it. With some extensions it could probably do more, but it's currently held up with needing to coordinate with T-infra to make sure the tests run in CI or bors. There's also the question of whether we even want to DIY this, or instead use a third-party service that's dedicated to this kind of testing.

We could probably do with a concerted effort to make sure we're testing the right things on rustdoc, too. It would probably be best to come up with more/better ideas for tests to add to rustdoc (even if they require a new kind of test) for me to be willing to sign off on featuring an item like this.

[GuillaumeGomez]

#[doc(include="...")]

As far as i know, this just needs someone to push for it? I haven't looked at it in a while.

Just needs stabilization iirc.

#[doc(cfg(...))]

If i remember right, there is an issue with platform-specific items being returned from functions, but otherwise, the core feature works fine?

Just like you said.

Intra-doc links

This probably needs the most work. There are still some spurious resolution errors being emitted, and we're still missing proper cross-crate resolution. We'll likely need to coordinate with someone from T-compiler to push this over the line.

It's a bit more than that. At first, we should try to clean it up a bit because it's getting a bit messy (even more considering this is a new feature!).

For the UI test PR, I talked about it with @simulcram. I explained to him our need and everything. The problem being that they don't have enough time to setup a server (I proposed to do it myself but didn't get answer on it so I guess they prefer to keep this in the infra team?).

That said, what does the team want to see from rustdoc in the coming year?

We added a lot of new features lately. And a lot of them are still unstable. I think we should at least give a definitive opinion to a few of them instead of just keeping them this way.

Another thing (linked to the previous one) I'd like to see is a global rustdoc cleanup (well, it has been started but having a global overview of it would be nice).

The last thing I'd want: a real separation between rustdoc output generation and the code. It'd allow us to handle multiple output kinds (I think very strongly about markdown right now).

None of these points is very difficult in itself, but they'd all take a lot of time. I think they're worth it though.

[nrc]

We added a lot of new features lately. And a lot of them are still unstable. I think we should at least give a definitive opinion to a few of them instead of just keeping them this way.

It's a good idea to keep on top of this, we have similar issues in the Cargo team. Maybe a good idea to have regular stabilisation triage or something similar

Another thing (linked to the previous one) I'd like to see is a global rustdoc cleanup

+1 Keeping on top of technical debt is important

The last thing I'd want: a real separation between rustdoc output generation and the code

Similarly, I'd love to see Rustdoc towards more standard html generation (handlebars or something, I'm not really up to date on specific libraries here). It wouldn't need to change anything radical like we've previously discussed, but just separating code from presentation would enable people with a more web background to contribute more and allow us to iterate faster on presentation ideas.

The other things on my mind for Rustdoc are using save-analysis for 'view source' (so we'd have something similar to cargo src or sourcegraph) and some kind of focused hacking on paper cuts - maybe an ergonomics hack week or something.

[steveklabnik]

We're gonna do the "blog posts" thing again this year for planning next year, and so I'm gonna think about this, and put some stuff in there. I'll link it back here whenever it gets posted.

[QuietMisdreavus]

Something else i remembered this morning was how, a long time ago, there was a discussion on trying to remove the notion of impl blocks from rustdoc's output, and instead print methods individually and include any impl-level information on each method.

For example, instead of the following:

impl<T> SomeStruct<T>

fn some_fn(&self) -> bool

Here's some docs for some_fn.

impl SomeStruct<()>

fn take_unit(self)

If your SomeStruct is on (), you can do this too.

impl<T: Copy> SomeTrait for SomeStruct<T>

fn trait_fn(&self)

Operates the trait.

You could do something like:

fn some_fn(&self) -> bool

Here's some docs for some_fn.

fn take_unit(self)

• Available on: SomeStruct<()>

If your SomeStruct is on (), you can do this too.

fn trait_fn(&self)

• use SomeTrait; (NB: SomeTrait would be hyperlinked here)
• where T: Copy

Operates the trait.

There may be problems when this is applied to certain traits with lots of methods (like Iterator) or on marker traits without any methods at all. It's kind of a big redesign for docs, but with some effort i think we could figure it out.

[QuietMisdreavus]

Doctests

While doctests are a powerful feature, they are still filled with some papercuts that would be worth focusing on for a few new features. Here are a few that i remember hearing about or thinking of:

• The ability to save/recover compiled doctests or the source that created them, for debugging (rustdoc: add the ability to build a doctest executable without running it rust-lang/rust#43030)
• The ability to run doctests with a "wrapper" script, akin to a "test runner" that i swear i've seen in cargo but can't find any more (rustdoc: accept a "test runner" argument to wrap around doctest executables rust-lang/rust#51228)
• The ability to compile doctests for a different target (rustdoc --test ignores --target rust-lang/rust#44404) (compounds with the above to make both way more useful)

There are probably other issues that i don't know about, too.

[QuietMisdreavus]

Source view

A couple smaller points here, but the source-view section of rustdoc output is kinda lackluster compared to the generated API docs. There's a high ceiling for how good a source browser can be (with a proportional amount of work required to do so) so there's a lot of room for improvement here. Here are just a couple ideas, one small, one big:

• Add a sidebar to the source view that shows the other source files in the crate. Right now, a source view is a dead-end page - you can't go anywhere from there. Adding just the ability to go to a different file without having to guess the module it corresponds to would help out.
• On the "high effort, high reward" end, we can integrate or clone cargo-src for a much more powerful view, a la the .NET Reference Source browser.
  • To integrate cargo-src directly we'd need a way to collect the compiler's analysis data, but i bet we already have some or all of the information on hand during doc collection. The question would be if there's a convenience method to spit out the JSON or if we need to mess with cargo-src itself so we can patch data to it directly.

[GuillaumeGomez]

It's true that source view hasn't received much updates for a (very very very) long time now. Should maybe be time that I start taking another look. :)

[QuietMisdreavus]

One update about #[doc(include)] - going from its tracking issue thread, it looks like we need to ensure we're reporting the correct file/line for doctest failures when the doctests come from included files.

And one about the source view - it looks like imperio has opened rust-lang/rust#55707 to implement the first point already! I'm reading through it right now, but it's exciting to be this close this quickly.

---

There are several good ideas going on here (with the promise of more when the open call for blog posts occurs, as @steveklabnik mentioned) but i think it's good to lay out the vague plan i have in mind.

• We can continue to collect ideas for the next couple weeks. I appreciate having this laid-back brainstorming session like this, because even if we turn down using an idea in the roadmap, it's still written down for future reference.
• Before the end of the year, i'd like to start narrowing down the list to 2-3 "big ideas" that we can get together in 2019. I already have my favorites (cargo-src and intra-doc links) but if someone else comes up with something high-impact and doable with our resources, i'm willing to hear it out.
  • After a conversation with @nrc recently, i think we can continue the cleanup/refactoring/test work we're doing as an ambient "continued maintenance" item, as well. It does no harm to include a statement to the effect of "continued cleanup and refactoring work, as well as improved testing of rustdoc itself, can also happen as we see fit", to give us an out to refactor things as necessary.

I'm not sure what the core team wants as its own roadmap timeline (i'm not sure how much consideration has been given to it, since the edition work has been ongoing) but i want to make sure we have something to talk about when that planning occurs.

---

A couple more related notes:

• Planning work for docs.rs can happen concurrently. I know @onur has some ideas in mind for structural changes, and discussions are ongoing about its hosting as we transition to hosting sponsored by the Rust Project. It may be prudent to open a new thread on this repo to talk just about docs.rs, to bring down the amount of cross-talk that can happen with semi-unrelated conversations happening in the same space.
• With the Rust All-Hands happening in February, we can use this list to plan out sessions or meetings that we want to happen there, and possibly begin to talk with other teams if we need to borrow time from them. (For example, one of the major blockers for intra-doc links, cross-crate resolution scopes, will probably require expertise from the compiler team, so we should bring this up with them soon if we want to pursue this. They're already beginning to plan out their focuses for the all-hands, so getting this on their radar will greatly help for getting their cooperation.)

[ishitatsuyuki]

I'd like to bring back a machine friendly output like JSON. Combined with some templating (either JavaScript or server-side), it can help docs.rs quite a lot.

@QuietMisdreavus has written a previous thread on this.

Also something that is mentioned in thread: what can we do to integrate it with save-analysis?

[QuietMisdreavus]

One more idea that keeps re-appearing, is some kind of effort to deal with the "file explosion" problem, where rustdoc needs to create two files for each item in a crate (one real file and one with an old-style name that redirects to the real one). With some crates, this can create thousands of files or more, which is a performance nightmare, especially in the presence of an anti-virus software or on certain filesystems.

One way that has been offered to deal with the problem is to radically re-architect how documentation is generated and viewed. This resulted in the "rustdoc rewrite" projects from the beginning of this year, where the idea is to use a machine-readable output (in this case, converted from the save-analysis JSON), then write a rich web application that does one file load - of the data file - and acts like a web server, reading information from the saved data and rendering pages as needed, without storing each item as a separate physical file.

The rustdoc rewrites faltered from the save-analysis data not having all the information that the current rustdoc needs, which goes back to the thought that @ishitatsuyuki mentioned: getting rustdoc to output the JSON itself. (Or extending save-analysis with this data, which may be the "cleaner" goal.) This way, we can sort of "let a thousand implementations bloom", and keep the "official" rustdoc around as long as we need.

---

I would like to propose a parallel effort to that, though: @killercup's static-filez. https://github.com/killercup/static-filez

He made the crate as a proof-of-concept to show how to have an archive of individually-compressed files be served through a web server directly, without having to extract and recompress them. Its docs and design issues explicitly reference using rustdoc output as a basis for these archives, so i think it would make an worthwhile experiment to "cut out the middle-man" and have rustdoc spit out one of these archives in the first place. (The complete plan for fully realizing this solution requires coordination with Cargo, but at least we can start there.)

[GuillaumeGomez]

One more idea that keeps re-appearing, is some kind of effort to deal with the "file explosion" problem, where rustdoc needs to create two files for each item in a crate (one real file and one with an old-style name that redirects to the real one). With some crates, this can create thousands of files or more, which is a performance nightmare, especially in the presence of an anti-virus software or on certain filesystems.

I saw that too. We should just remove those duplicates. That'd make the whole situation almost twice better thanks to this.

The rustdoc rewrites faltered from the save-analysis data not having all the information that the current rustdoc needs, which goes back to the thought that @ishitatsuyuki mentioned: getting rustdoc to output the JSON itself. (Or extending save-analysis with this data, which may be the "cleaner" goal.) This way, we can sort of "let a thousand implementations bloom", and keep the "official" rustdoc around as long as we need.

Adding alternative outputs would be a good start. JSON and markdown would be nice as a start. Also, considering that some people don't want to enable JS, we're a bit limited on this side, otherwise I would have put more JS in pages in order to generate them from those data instead of huge and ugly HTML.

I would like to propose a parallel effort to that, though: @killercup's static-filez. https://github.com/killercup/static-filez

I talked about it with him directly and to sum up what came out: that'd be nice to have it alongside with a different rustup option/cargo doc run so you can have whatever pleases you the most. I like the fact that I don't need a server running to have access to the docs but I'm fine having both.