[build] Normal & Diagnostic Log Verbosity, at once! #1808
Conversation
Fixes: dotnet#1792 We have a bit of a conundrum: when things break, the diagnostic build logs are *essential* to tracking down *why* something broke. However, the sheer *verbosity* of those diagnostic logs makes it difficult to determine *what* broke; it's very easy to get lost. Additionally, the gigantic log files make the **Console Output** Jenkins link nearly useless, as most web browsers will cease up when attempting to download ~1GB log files. Split the difference: introduce a new `$(call MSBUILD_BINLOG)` syntax which creates an [MSBuild binary log][0] file, which is like diagnostic output *but even moreso*, while *also* overriding the console logging to `/v:normal`. [0]: https://github.com/Microsoft/msbuild/wiki/Binary-Log This should allow the **Console Output** link to be a bit more reasoanble, and if something goes wrong we can download the separate `*.binlog` files for further analysis. MSBuild binary log files can be read with MSBuild itself: msbuild /v:diag bin/BuildDebug/msbuild-*.binlog Note: above command will actually fail; you can only print one log file at a time with that syntax. The created log files follow the pattern `msbuild-EPOCH-NAME`, where EPOCH is the `time_t`/Unix epoch value *at the time the command was executed*. This should allow filenames to be non-colliding, and allow plausibly sane sorting behavior. (Maybe.)
|
This PR/commit does not fix #1792 by itself. What is important there is to get CI build results readable. What is more important and kindful for developers is that there is a solid and supported way to run individual tests to diagnose the issues. If each contributor is asked to run "hacky" way to run individual tests and be blamed if they get false results by adding hacks, then those contributors are pissed off. It does not have to be done within this PR, but if it's not achieved then we'll have to keep the issue open. |
|
Build is failing and because of the exact issue on the context I cannot see why it's failing :( |
|
Your concern was strictly that the build log was too big:
The log file is now smaller; the build log for this PR is ~95MB. The previous build log size when a non-full mono rebuild was involved was 206MB; we're less than half the size, and much of that output is the mono build. As such, I'm now able to click the Console Output link and view it within 20 seconds. (Yay?)
You still need to read the log output: line 979566 -- 310 lines from the end of the file -- contains the error: Is it too much to go to the end of the file and search backwards for As for why |
That wasn't mentioned in Issue #1792, and we do have documentation on running individual tests, though apparently we need to document running a single MSBuild unit test. @dellis1972: would you be able to update the docs to include running a single MSBuild unit test fixture and/or method? |
|
@jonpryor I will |
|
I mean, build individual tests and get separate test results on the CI builds, instead of forcing contributors read all those annoyingly irrelevant build logs. |
You mean like the Test Result link on Jenkins? If a unit test fails, you don't need to read Console Output to determine the error. For example: https://jenkins.mono-project.com/job/xamarin-android-pr-builder/3315/ > Additionally, the Console Output link won't even show the build log for that failing test; you'd have to use |
|
That aside, this PR appears to be incomplete: some portions of the output are not "normal" verbosity, but "diagnostic" verbosity: None of that should appear in "normal" verbosity. Looks like there are some additional spots to fix, e.g.: https://github.com/xamarin/xamarin-android/blob/391970420f92df5923bd92118ca1d78e228303bf/build-tools/scripts/RunTests.targets#L84 |
|
Where can I find such a "Test Result link" on https://jenkins.mono-project.com/job/xamarin-anroid-linux-pr-builder/2993/ ? (it is the context of the issue #1792 I reported) (Although let's make sure that separate build logs don't have to be part of this PR.) |
There isn't one. Our builds have three possible states:
The build you linked to is Red. Something failed to build. Consequently, no tests were run. As such, there is not a "test result link," nor can there be one. Reviewing the log for that build, I see: It's 51 lines from the end of the file; it is not hard to find this error message. (It is also repeated 5 times for various assemblies.) The cause for the |
|
build |
Fixes: #1792
We have a bit of a conundrum: when things break, the diagnostic build
logs are essential to tracking down why something broke.
However, the sheer verbosity of those diagnostic logs makes it
difficult to determine what broke; it's very easy to get lost.
Additionally, the gigantic log files make the Console Output
Jenkins link nearly useless, as most web browsers will cease up when
attempting to download ~1GB log files.
Split the difference: introduce a new
$(call MSBUILD_BINLOG)syntaxwhich creates an MSBuild binary log file, which is like
diagnostic output but even moreso, while also overriding the
console logging to
/v:normal.This should allow the Console Output link to be a bit more
reasoanble, and if something goes wrong we can download the separate
*.binlogfiles for further analysis.MSBuild binary log files can be read with MSBuild itself:
Note: above command will actually fail; you can only print one log
file at a time with that syntax.
The created log files follow the pattern
msbuild-EPOCH-NAME, whereEPOCH is the
time_t/Unix epoch value at the time the command wasexecuted. This should allow filenames to be non-colliding, and allow
plausibly sane sorting behavior. (Maybe.)