Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some notes on testing trimming #48333

Merged
merged 2 commits into from
May 22, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/Trimming.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,65 @@ This should update the `WarningSuppressions.xml` files associated with projects.
- LegacyRouteTableFactory.<>c.{Create}b__2_1(System.Reflection.Assembly)
+ LegacyRouteTableFactory.<>c.<Create>b__2_1(System.Reflection.Assembly)
```

## Validate trimming behavior

We have infrastructure for writing tests to validate trimming behavior.
The two most common tasks are confirming that functionality still works after trimming and confirming that a particular type or member was not preserved.

### Infrastructure

Because trimming is only performed during publish, the tests can't be normal xunit tests.
Instead, we have a system that wraps individual source files in appropriately configured projects and publishes them as standalone executables.
These executables are then run and the test outcome is based on the exit code.
The tests are powered by [trimmingTests.targets](..\eng\testing\linker\trimmingTests.targets), which is based on corresponding functionality in https://github.com/dotnet/runtime.
amcasey marked this conversation as resolved.
Show resolved Hide resolved

### Adding a new test project

Adding a new test project is very simple - just call it *.TrimmingTests.proj and give it a body like
```xml
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<TestConsoleAppSourceFiles Include="MyTest.cs">
</ItemGroup>
</Project>
```

Unfortunately, there's no good way to specify which aspnetcore projects the tests depend upon so, to avoid dependency ordering problems, it's easiest to specify that they should be built after the main build.
Do this by adding an entry to [RequiresDelayedBuildProjects.props](..\eng\RequiresDelayedBuildProjects.props).
amcasey marked this conversation as resolved.
Show resolved Hide resolved

### Adding a new test

A test is just a C# file with a main method (or top-level statements, if you prefer).
Write a program that either should keep working after trimming or that validates a particular type/member has been trimmed (e.g. using `Type.GetType`).
If things behave as expected exit with code `100` (a magic number intended to prevent accidental passes).
Any other result - a different exit code or a crash - will count as a failure.

Now that you have a test program, add a `TestConsoleAppSourceFiles` item to the corresponding test project.

If you have additional files, typically because you want to share code between tests, you can use `AdditionalSourceFiles`.
If you want to en/disable a particular feature switch, you can use `EnabledFeatureSwitches`/`DisabledFeatureSwitches`.

### Running tests

To run the tests locally, build the projects it depends on (it's usually easiest to just build the whole repo) and then run
```
.dotnet\dotnet.exe test path\to\project\folder\
```

Alternatively, you can use `Activate.ps1` to ensure you're using the right dotnet binary.

### Native AOT

Native AOT testing is substantially the same - just name your project *.NativeAotTests.proj, rather than *.TrimmingTests.proj.

### Tips

- It's easier to author your test as a standalone Native AOT app and then copy the final code into the test (don't forget to add the copyright header).
The two main advantages are that building is much faster and you get console output that you can use to help debug your test.
- If you are using AdditionalSourceFiles and you need to make a change to one of the additional files, you will also need to update the test file itself - just editing the additional file _won't trigger a rebuild_ on the next run.
- `Type.GetType` is known to the trimmer, so you'll have to (a) make it impossible to statically determine what argument you're passing (so that it isn't preserved) and (b) suppress the resulting warning: `[UnconditionalSuppressMessage("ReflectionAnalysis", "IL2057:UnrecognizedReflectionPattern", Justification = "Using GetType to test trimming")]`
- Pass an assembly-qualified name (`"ns.type, assembly.name"`) to `Type.GetType` as it searches only the executing assembly and corlib by default.
- There's no output indicating that a test has passed, so it's important to check that it's actually running (e.g. by inserting an artificial failure).
- If your test depends on any JSON serialization functionality, you'll probably need to disable the `System.Text.Json.JsonSerializer.IsReflectionEnabledByDefault` feature switch.
- If your test validates that a type is being trimmed, it's good practice to write a dual test (sharing the same helpers) that validates that it is preserved (in a different scenario, of course) so that the trimming test doesn't accidentally pass because of (e.g.) a typo in the type name passed to `GetType`.