Huge build logs make contributions impossible #1792
Assignees
Labels
Area: xamarin-android Build
Issues building the xamarin-android repo *itself*.
Comments
atsushieno
added
the
Area: xamarin-android Build
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this issue
Jun 12, 2018
Fixes: dotnet#1792 We have a bit of a conundrum: when things break, the diagnostic build logs are *essential* to tracking down *why* something broke. However, the sheer *verbosity* of those diagnostic logs makes it difficult to determine *what* broke; it's very easy to get lost. Additionally, the gigantic log files make the **Console Output** Jenkins link nearly useless, as most web browsers will cease up when attempting to download ~1GB log files. Split the difference: introduce a new `$(call MSBUILD_BINLOG)` syntax which creates an [MSBuild binary log][0] file, which is like diagnostic output *but even moreso*, while *also* overriding the console logging to `/v:normal`. [0]: https://github.com/Microsoft/msbuild/wiki/Binary-Log This should allow the **Console Output** link to be a bit more reasoanble, and if something goes wrong we can download the separate `*.binlog` files for further analysis. MSBuild binary log files can be read with MSBuild itself: msbuild /v:diag bin/BuildDebug/msbuild-*.binlog Note: above command will actually fail; you can only print one log file at a time with that syntax. The created log files follow the pattern `msbuild-EPOCH-NAME`, where EPOCH is the `time_t`/Unix epoch value *at the time the command was executed*. This should allow filenames to be non-colliding, and allow plausibly sane sorting behavior. (Maybe.)
atsushieno
pushed a commit
that referenced
this issue
Jun 14, 2018
Fixes: #1792 We have a bit of a conundrum: when things break, the diagnostic build logs are *essential* to tracking down *why* something broke. However, the sheer *verbosity* of those diagnostic logs makes it difficult to determine *what* broke; it's very easy to get lost. Additionally, the gigantic log files make the **Console Output** Jenkins link nearly useless, as most web browsers will cease up when attempting to download ~1GB log files. Split the difference: introduce a new `$(call MSBUILD_BINLOG)` syntax which creates an [MSBuild binary log][0] file, which is like diagnostic output *but even moreso*, while *also* overriding the console logging to `/v:normal`. [0]: https://github.com/Microsoft/msbuild/wiki/Binary-Log This should allow the **Console Output** link to be a bit more reasoanble, and if something goes wrong we can download the separate `*.binlog` files for further analysis. MSBuild binary log files can be read with MSBuild itself: msbuild /v:diag bin/BuildDebug/msbuild-*.binlog Note: above command will actually fail; you can only print one log file at a time with that syntax. The created log files follow the pattern `msbuild-EPOCH-NAME`, where EPOCH is the `time_t`/Unix epoch value *at the time the command was executed*. This should allow filenames to be non-colliding, and allow plausibly sane sorting behavior. (Maybe.)
|
As commented on #1808 it cannot be closed only by that PR. |
Closed
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this issue
Jul 25, 2018
Context: dotnet#1792 Context: 944092d99540bb9a6106175404c17cf4fddce394 Commit 944092d9 updated the "normal" build log process, as used by Jenkins/etc., so that Console output received "normal" build verbosity messages, while MSBuild `.binlog` files would be generated so that "diagnostic" build logs could be produced "later," as necessary. Unfortunately, commit 944092d9 didn't update unit test builds. Consequently, `make run-apk-tests`/etc. would emit *diagnostic* build logs to the Console, meaning instead of the previous ~1GB log file, we instead had a "mere" ~400MB instead. Update test execution so that they follow the spirit of 944092d9: `.binlog` files for diagnostic output, with "normal" output printed to the console, resulting in smaller Jenkins log files.
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this issue
Jul 25, 2018
Context: dotnet#1792 Context: 987a05f Commit 987a05f updated the "normal" build log process, as used by Jenkins/etc., so that Console output received "normal" build verbosity messages, while MSBuild `.binlog` files would be generated so that "diagnostic" build logs could be produced "later," as necessary. Unfortunately, commit 987a05f didn't update unit test builds. Consequently, `make run-apk-tests`/etc. would emit *diagnostic* build logs to the Console, meaning instead of the previous ~1GB log file, we instead had a "mere" ~400MB instead. Update test execution so that they follow the spirit of 987a05f: `.binlog` files for diagnostic output, with "normal" output printed to the console, resulting in smaller Jenkins log files.
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this issue
Jul 26, 2018
Context: dotnet#1792 Context: 987a05f Commit 987a05f updated the "normal" build log process, as used by Jenkins/etc., so that Console output received "normal" build verbosity messages, while MSBuild `.binlog` files would be generated so that "diagnostic" build logs could be produced "later," as necessary. Unfortunately, commit 987a05f didn't update unit test builds. Consequently, `make run-apk-tests`/etc. would emit *diagnostic* build logs to the Console, meaning instead of the previous ~1GB log file, we instead had a "mere" ~400MB instead. Update test execution so that they follow the spirit of 987a05f: `.binlog` files for diagnostic output, with "normal" output printed to the console, resulting in smaller Jenkins log files.
jonpryor
added a commit
that referenced
this issue
Oct 11, 2018
Context: #1792 Context: 987a05f Commit 987a05f updated the "normal" build log process, as used by Jenkins/etc., so that Console output received "normal" build verbosity messages, while MSBuild `.binlog` files would be generated so that "diagnostic" build logs could be produced "later," as necessary. Unfortunately, commit 987a05f didn't update unit test builds. Consequently, `make run-apk-tests`/etc. would emit *diagnostic* build logs to the Console, meaning instead of the previous ~1GB log file, we instead had a "mere" ~400MB instead. Update test execution so that they follow the spirit of 987a05f: `.binlog` files for diagnostic output, with "normal" output printed to the console, resulting in smaller Jenkins log files -- ~26MB now.
|
Jenkins is no longer used, so I think this issue is obsolete. |
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Here are what happens:
Whether it is a Chrome issue or not does not matter. What is important here is that Xamarin does not give sane way to diagnose any blocking problem for contributors (and of course our own) pull requests.
We have to fix this either 1) by reducing insane amount of build logs, or 2) provide any way to diagnose logs.
Actually 2) is not really a viable solution. Xamarin cannot ask contributors to read hundreds of megabytes of single build logs to investigate.
When we were using old CI server (wrench) it had never occurred because each build steps were split and we had to read only relevant log parts.
The text was updated successfully, but these errors were encountered: