From 8a766564351cf9a2305269aeb28e2db658d701cf Mon Sep 17 00:00:00 2001 From: Tom Tromey Date: Wed, 1 Aug 2018 11:36:28 -0600 Subject: [PATCH] Link compiler test documentation to rustc-guide Update the compiler test documentation to point to the relevant rustc-guide page. --- src/test/COMPILER_TESTS.md | 186 +------------------------------------ 1 file changed, 2 insertions(+), 184 deletions(-) diff --git a/src/test/COMPILER_TESTS.md b/src/test/COMPILER_TESTS.md index 7dabb1bddea77..81a46ea0fe718 100644 --- a/src/test/COMPILER_TESTS.md +++ b/src/test/COMPILER_TESTS.md @@ -1,186 +1,4 @@ # Compiler Test Documentation -In the Rust project, we use a special set of commands embedded in -comments to test the Rust compiler. There are two groups of commands: - -1. Header commands -2. Error info commands - -Both types of commands are inside comments, but header commands should -be in a comment before any code. - -## Summary of Error Info Commands - -Error commands specify something about certain lines of the -program. They tell the test what kind of error and what message you -are expecting. - -* `~`: Associates the following error level and message with the - current line -* `~|`: Associates the following error level and message with the same - line as the previous comment -* `~^`: Associates the following error level and message with the - previous line. Each caret (`^`) that you add adds a line to this, so - `~^^^^^^^` is seven lines up. - -The error levels that you can have are: - -1. `ERROR` -2. `WARNING` -3. `NOTE` -4. `HELP` and `SUGGESTION`* - -\* **Note**: `SUGGESTION` must follow immediately after `HELP`. - -## Summary of Header Commands - -Header commands specify something about the entire test file as a -whole. They are normally put right after the copyright comment, e.g.: - -```Rust -// Copyright blah blah blah -// except according to those terms. - -// ignore-test This doesn't actually work -``` - -### Ignoring tests - -These are used to ignore the test in some situations, which means the test won't -be compiled or run. - -* `ignore-X` where `X` is a target detail or stage will ignore the test accordingly (see below) -* `ignore-pretty` will not compile the pretty-printed test (this is done to test the pretty-printer, but might not always work) -* `ignore-test` always ignores the test -* `ignore-lldb` and `ignore-gdb` will skip a debuginfo test on that debugger. - -`only-X` is the opposite. The test will run only when `X` matches. - -Some examples of `X` in `ignore-X`: - -* Architecture: `aarch64`, `arm`, `asmjs`, `mips`, `wasm32`, `x86_64`, `x86`, ... -* OS: `android`, `emscripten`, `freebsd`, `ios`, `linux`, `macos`, `windows`, ... -* Environment (fourth word of the target triple): `gnu`, `msvc`, `musl`. -* Pointer width: `32bit`, `64bit`. -* Stage: `stage0`, `stage1`, `stage2`. - -### Other Header Commands - -* `min-{gdb,lldb}-version` -* `min-llvm-version` -* `compile-pass` for UI tests, indicates that the test is supposed - to compile, as opposed to the default where the test is supposed to error out. -* `compile-flags` passes extra command-line args to the compiler, - e.g. `compile-flags -g` which forces debuginfo to be enabled. -* `should-fail` indicates that the test should fail; used for "meta testing", - where we test the compiletest program itself to check that it will generate - errors in appropriate scenarios. This header is ignored for pretty-printer tests. -* `gate-test-X` where `X` is a feature marks the test as "gate test" for feature X. - Such tests are supposed to ensure that the compiler errors when usage of a gated - feature is attempted without the proper `#![feature(X)]` tag. - Each unstable lang feature is required to have a gate test. - -## Revisions - -Certain classes of tests support "revisions" (as of the time of this -writing, this includes run-pass, compile-fail, run-fail, and -incremental, though incremental tests are somewhat -different). Revisions allow a single test file to be used for multiple -tests. This is done by adding a special header at the top of the file: - -``` -// revisions: foo bar baz -``` - -This will result in the test being compiled (and tested) three times, -once with `--cfg foo`, once with `--cfg bar`, and once with `--cfg -baz`. You can therefore use `#[cfg(foo)]` etc within the test to tweak -each of these results. - -You can also customize headers and expected error messages to a particular -revision. To do this, add `[foo]` (or `bar`, `baz`, etc) after the `//` -comment, like so: - -``` -// A flag to pass in only for cfg `foo`: -//[foo]compile-flags: -Z verbose - -#[cfg(foo)] -fn test_foo() { - let x: usize = 32_u32; //[foo]~ ERROR mismatched types -} -``` - -Note that not all headers have meaning when customized to a revision. -For example, the `ignore-test` header (and all "ignore" headers) -currently only apply to the test as a whole, not to particular -revisions. The only headers that are intended to really work when -customized to a revision are error patterns and compiler flags. - -## Guide to the UI Tests - -The UI tests are intended to capture the compiler's complete output, -so that we can test all aspects of the presentation. They work by -compiling a file (e.g., `ui/hello_world/main.rs`), capturing the output, -and then applying some normalization (see below). This normalized -result is then compared against reference files named -`ui/hello_world/main.stderr` and `ui/hello_world/main.stdout`. If either of -those files doesn't exist, the output must be empty. If the test run -fails, we will print out the current output, but it is also saved in -`build//test/ui/hello_world/main.stdout` (this path is -printed as part of the test failure message), so you can run `diff` and -so forth. - -Normally, the test-runner checks that UI tests fail compilation. If you want -to do a UI test for code that *compiles* (e.g. to test warnings, or if you -have a collection of tests, only some of which error out), you can use the -`// compile-pass` header command to have the test runner instead -check that the test compiles successfully. - -### Editing and updating the reference files - -If you have changed the compiler's output intentionally, or you are -making a new test, you can pass `--bless` to the command you used to -run the tests. This will then copy over the files -from the build directory and use them as the new reference. - -### Normalization - -The normalization applied is aimed at eliminating output difference -between platforms, mainly about filenames: - -- the test directory is replaced with `$DIR` -- all backslashes (`\`) are converted to forward slashes (`/`) (for Windows) -- all CR LF newlines are converted to LF - -Sometimes these built-in normalizations are not enough. In such cases, you -may provide custom normalization rules using the header commands, e.g. - -``` -// normalize-stdout-test: "foo" -> "bar" -// normalize-stderr-32bit: "fn\(\) \(32 bits\)" -> "fn\(\) \($$PTR bits\)" -// normalize-stderr-64bit: "fn\(\) \(64 bits\)" -> "fn\(\) \($$PTR bits\)" -``` - -This tells the test, on 32-bit platforms, whenever the compiler writes -`fn() (32 bits)` to stderr, it should be normalized to read `fn() ($PTR bits)` -instead. Similar for 64-bit. The replacement is performed by regexes using -default regex flavor provided by `regex` crate. - -The corresponding reference file will use the normalized output to test both -32-bit and 64-bit platforms: - -``` -... - | - = note: source type: fn() ($PTR bits) - = note: target type: u16 (16 bits) -... -``` - -Please see `ui/transmute/main.rs` and `.stderr` for a concrete usage example. - -Besides `normalize-stderr-32bit` and `-64bit`, one may use any target -information or stage supported by `ignore-X` here as well (e.g. -`normalize-stderr-windows` or simply `normalize-stderr-test` for unconditional -replacement). +Documentation the compiler testing framework has moved to +[the rustc guide](https://rust-lang-nursery.github.io/rustc-guide/tests/intro.html).