[CI] Create the initial stage to start running APIScan. #19828
Conversation
APIScan is slow (as per their own documentation). To ensure that the extra check does not slow our CI we create a matrix that will run APIScan in each of the platforms to later be uploaded to TSA via a funnel job.
mandel-macaque
changed the title
| INCLUDE: $(INCLUDE_DOTNET_IOS) | ||
| PLATFORM: iOS | ||
| SECRET: $(ApiScanSecretiOS) | ||
| VERSION: 17.0 |
There was a problem hiding this comment.
The VERSION field doesn't seem to be used (which is probably good, because it's outdated already... should be 17.2).
There was a problem hiding this comment.
@rolfbjarne it will be used, but we will get it from the config step. It is a place holder atm. APIScan takes the version of the product as a parameter so that we can identify the actual problem.
| tvOS: | ||
| INCLUDE: $(INCLUDE_DOTNET_TVOS) | ||
| PLATFORM: tvOS | ||
| SECRET: $(ApiScanSecretiOS) | ||
| VERSION: 17.0 | ||
| macCatalyst: | ||
| INCLUDE: $(INCLUDE_DOTNET_MACCATALYST) | ||
| PLATFORM: macCatalyst | ||
| SECRET: $(ApiScanSecretiOS) | ||
| VERSION: 17.0 | ||
| macOS: | ||
| INCLUDE: $(INCLUDE_DOTNET_MACOS) | ||
| PLATFORM: macOS | ||
| SECRET: $(ApiScanSecretmacOS) | ||
| VERSION: 14.0 |
There was a problem hiding this comment.
I'm not very fond of hardcoding the list of platforms in yet another place (visionOS...), so would it be possible to generate this somehow? The DOTNET_PLATFORMS variable (stageDependencies.configure_build.configure.outputs['configure_platforms.DOTNET_PLATFORMS']) should contain the list of enabled platforms.
There was a problem hiding this comment.
lets see if I can do that, but there are certain problem with using a runtime variable in a template.
There was a problem hiding this comment.
@rolfbjarne the change you are asking, as we have everything is impossible. With he yaml template synstax we have 3 diff stages:
${{ }}: This is a template expression. It gets executed BEFORE anything runs and expands the data.$[]: Expanded at runtime.
A template cannot use a value that is generated and runtime. If we really wanted to get to the point in which we can generate things like the one you suggest, we should do a two step, in which we generate a yaml file with the variables/parameters needed by the templates and then generated. Else we can move to use template parameter form the pipeline and push them down. It will mean we need to update those params in a single place.
AZP does not give us many options. If we want to go that route, we need to move the new platforms param I added to the entry level yaml for both ci and pr.
There was a problem hiding this comment.
You can generate the matrix, this is what the test splitting PR does:
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mandel-macaque
force-pushed
the
download-workloads-apiscan
branch
2 times, most recently
from
f63b675 to
b77f489
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mandel-macaque
force-pushed
the
download-workloads-apiscan
branch
2 times, most recently
from
0843b4c to
cc6523c
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mandel-macaque
force-pushed
the
download-workloads-apiscan
branch
from
cc6523c to
c57b9e5
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mandel-macaque
force-pushed
the
download-workloads-apiscan
branch
3 times, most recently
from
c890229 to
1794bd4
Compare
mandel-macaque
force-pushed
the
download-workloads-apiscan
branch
from
1794bd4 to
bcaa9d6
Compare
| commit: ${{ parameters.commit }} | ||
|
|
||
| - pwsh: | | ||
| Write-Output "Performing APISCan for $(PLATFORM) using tenant $(TENANT) client id $(CLIENT_ID) and secret $(SECRET)." |
There was a problem hiding this comment.
Should the secret be displayed like this?
There was a problem hiding this comment.
Devops is super smart (for once), and knows not to display them:
📚 [PR Build] Artifacts 📚Packages generatedView packagesPipeline on Agent |
💻 [PR Build] Tests on macOS M1 - Mac Ventura (13.0) passed 💻✅ All tests on macOS M1 - Mac Ventura (13.0) passed. Pipeline on Agent |
💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻✅ All tests on macOS M1 - Mac Big Sur (11.5) passed. Pipeline on Agent |
💻 [CI Build] Windows Integration Tests passed 💻✅ All Windows Integration Tests passed. Pipeline on Agent |
✅ API diff for current PR / commitLegacy Xamarin (No breaking changes)
NET (empty diffs)
✅ API diff vs stableLegacy Xamarin (No breaking changes).NET (No breaking changes)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
🚀 [CI Build] Test results 🚀Test results✅ All tests passed on VSTS: simulator tests. 🎉 All 170 tests passed 🎉 Tests counts✅ cecil: All 1 tests passed. Html Report (VSDrops) Download Pipeline on Agent |
| 'tvOS': 'ApiScanSecretiOS', | ||
| 'macCatalyst': 'ApiScanSecretiOS', |
There was a problem hiding this comment.
Should there be separate secrets for tvOS and Mac Catalyst?
There was a problem hiding this comment.
Actually, don't longer need that
| variables: | ||
| ${{ each p in parameters.platforms }}: | ||
| INCLUDE_DOTNET_${{ upper(p.key) }}: $[ stageDependencies.configure_build.configure.outputs['configure_platforms.INCLUDE_DOTNET_${{ upper(p.key)}}'] ] | ||
| INCLUDE_DOTNET_${{ upper(p.key) }}_VERSION: 17.0 |
There was a problem hiding this comment.
17.0 is already passed... any way to not hardcode this value? It's also wrong for macOS.
There was a problem hiding this comment.
Same, I need cleaning
…rin#19828)" This reverts commit b7f4bef.
…n. (xamarin#19828)" (xamarin#20056)" This reverts commit 02947b2.
APIScan is slow (as per their own documentation). To ensure that the extra check does not slow our CI we create a matrix that will run APIScan in each of the platforms to later be uploaded to TSA via a funnel job.