SuperPMI: Improve superpmi-diffs summarization #80719
Comments
BruceForstall
added
the
area-CodeGen-coreclr
|
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak Issue DetailsCurrently, summarization is done for each AzDO job that spawns all the Helix work items. Thus, we end up with a section for Windows x64 and one for Windows x86 -- and each section contains all the runs that happened in that section (e.g., in the "Windows x86" section, there are both asmdiffs and tpdiff for both Windows x86 and Linux arm). It was implemented this way for convenience, but the output summarization could be better, and more integrated. There would be a "Summarization" AzDO job that depends on all the jobs that compute the diffs, and it should take all the results and compute the final summary file that becomes the "Extensions" page (via the magic category:eng-sys
|
|
Related: #80718 |
…machine Currently, we use a win-x86 machine for x86/arm work and a win-x64 machine for x64/arm64 work. The AzDO machine doesn't do very much: it downloads files, sets up the Helix correlation and payload directories, and constructs the set of Helix work items to run. Splitting into two different AzDO machines is an artifact of our YML files using a very complicated set of templates for setting the Helix queues. However, we can always use just a single win-x64 Helix queue for everything, since we do all the work using cross-compilers. Doing all the work on one AzDO machine would make it simpler to have integrated diff summarization. See dotnet#80719. However, even so it might be easier to create integrated summarization by creating a "summarize" AzDO job that depends on the others. This would also make it easier to enable native (not cross-compiler) replay, if desired.
|
Fixed by #106759 |
Currently, summarization is done for each AzDO job that spawns all the Helix work items. Thus, we end up with a section for Windows x64 and one for Windows x86 -- and each section contains all the runs that happened in that section (e.g., in the "Windows x86" section, there are both asmdiffs and tpdiff for both Windows x86 and Linux arm).
It was implemented this way for convenience, but the output summarization could be better, and more integrated.
There would be a "Summarization" AzDO job that depends on all the jobs that compute the diffs, and it should take all the results and compute the final summary file that becomes the "Extensions" page (via the magic
##vso[task.uploadsummary]tag). The superpmi_diffs_summarize.py script would run in this job.Alternatively, there could be a single AzDO job that kicks off all the Helix jobs. It would have to depend on all the builds, so starting the Helix work would be unnecessarily waiting on builds that not all Helix work requires. However, it might be much simpler to implement better summarization with a single AzDO job, and be more robust than synchronizing multiple "work" jobs.
category:eng-sys
theme:super-pmi
skill-level:intermediate
cost:medium
impact:small
The text was updated successfully, but these errors were encountered: