tests: Add htmldocck.py script for the use of Rustdoc tests.

[lifthrasiir]

The script is intended as a tool for doing every sort of verifications amenable to Rustdoc's HTML output. For example, link checkers would go to this script. It already parses HTML into a document tree form (with a slight caveat), so future tests can make use of it.

As an example, relevant rustdoc-* run-make tests have been updated to use htmldocck.py and got their verify.sh removed. In the future they may go to a dedicated directory with htmldocck running by default. The detailed explanation of test scripts is provided as a docstring of htmldocck.

cc #19723

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nikomatsakis (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see CONTRIBUTING.md for more information.

[brson]

@brson ❤️ tests

[tomjakubowski]

This is great! Hopefully this will help prevent a lot of regressions in rustdocs when things change internally by making it easier to write tests.

Will we be able to test cross-crate docs this way too? Specifically I mean checking that where clauses, lifetimes, and things like that are preserved when using pub use across crates.

[lifthrasiir]

@tomjakubowski In principle yes, one can invoke htmldocck multiple times with different template files. The script can also be updated to support multiple templates in order to avoid redundant works (e.g. they do not share parsed file caches).

[lifthrasiir]

Should htmldocck support a CSS selector syntax instead? XPath is what we have with a stock Python, but ElementTree's XPath support is already wanky that the script has some workaround. CSS-to-XPath translation is possible for useful subsets of them, but I'm concerned about the complexity of testing code.

[alexcrichton]

This... is amazing! Thanks so much for doing this @lifthrasiir!

Some thoughts:

• I want to land this ASAP :), all further thoughts are purely wishlist points
• Making a completely separate top-level src/test/rustdoc directory and integrating this into compiletest would be a pretty awesome step forward as well. It would solve @tomjakubowski's concern of cross-crate tests (via // aux-build) and would also give us parallel testing, non-makefile dependencies, and filtering all by default.
• I personally know very little about XPath, so I'd prefer to use CSS selectors, but I'll leave it up to you as to whether it's worth it (either way is totally fine by me).

If you're comfortable landing this now, I don't mind getting this in-tree ASAP to start requesting regression tests for rustdoc :)

[alexcrichton]

I can also help out with any form of integration with compiletest, I've driven around in there few times before!

[lifthrasiir]

@alexcrichton I'll try a little bit with CSS-to-XPath converter this night, but I don't mind the immediate merging.

[lifthrasiir]

Okay, I seriously tried that. Actually, it was not that hard to get a functional CSS parser in about 50 or 60 lines of additional code, but the resulting CSS selector will be severely limited since the partial matching of classes won't work (e.g. .foo will match class="foo bar" while [@class="foo"] won't). I gave up with the possibility of using CSS selectors, so I do not object merging it as is now.

[alexcrichton]

Alright, thanks for investigating! We can continue to iterate a bit in-tree

[alexcrichton]

@bors: r+ 2c1d1fb

[tomjakubowski]

@lifthrasiir yeah, XPath unfortunately doesn't have a nice shortcut for matching on whitespace-delimited tokens in an attribute (like CSS3's ~= selector). This could be useful if you don't already know this trick: http://stackoverflow.com/a/1390680

[lifthrasiir]

@tomjakubowski ElementTree does not support any function besides from last(), so that wouldn't work anyway. ;_;

[bors]

⌛ Testing commit 2c1d1fb with merge 3bf41da...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-win-32-nopt-t, auto-win-32-opt, auto-win-64-nopt-t, auto-win-64-opt

[sinistersnare]

It might be a good idea to make the python code here forward compatible with python3. Always good to prepare for the inevitable.