Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Create the initial stage to start running APIScan. #19828

Merged
merged 7 commits into from
Feb 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions tools/devops/automation/scripts/Governance.psm1
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
class APIScanConfiguration {
[string[]] $enabledPlatforms

APIScanConfiguration (
[string[]] $enabledPlatforms) {
$this.enabledPlatforms = $enabledPlatforms
}

[string] Create() {
$vars = [ordered]@{}
Write-Host "enabledPlatforms: $($this.enabledPlatforms)"

foreach ($platform in $this.enabledPlatforms) {
# dictionary with the secrets needed by each matrix
$platformVars = [ordered]@{
CLIENT_ID = $Env:API_SCAN_CLIENT_ID;
TENANT = $Env:API_SCAN_TENANT;
SECRET = "`$(API_SCAN_SECRET_$($platform.ToUpper()))";
PLATFORM = $platform.ToUpper();
}
$vars[$platform] = $platformVars
}

return $vars | ConvertTo-Json
}

}

function Get-APIScanConfiguration {
param
(
[Parameter(Mandatory)]
[string]
[AllowEmptyString()]
$EnabledPlatforms
)

$arrEnabledPlatforms = -split $EnabledPlatforms
$config = [APIScanConfiguration]::new($arrEnabledPlatforms)
return $config.Create()
}

# export public functions, other functions are private and should not be used outside the module.
Export-ModuleMember -Function Get-APIScanConfiguration
1 change: 1 addition & 0 deletions tools/devops/automation/scripts/MaciosCI.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ ScriptsToProcess = @(
NestedModules = @(
'Artifacts.psm1',
'GitHub.psm1',
'Governance.psm1',
'MLaunch.psm1',
'RemoteMac.psm1',
'StaticPages.psm1',
Expand Down
13 changes: 12 additions & 1 deletion tools/devops/automation/templates/common/configure.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,22 @@ steps:
$testMatrix = $testMatrix | ConvertFrom-Json | ConvertTo-Json -Compress
Write-Host "##vso[task.setvariable variable=TEST_MATRIX;isOutput=true]$testMatrix"
name: test_matrix
displayName: 'Create strategy matrix'
displayName: 'Create tests strategy matrix'
env:
TEST_CONFIGURATIONS: ${{ convertToJson(parameters.testConfigurations) }}
SUPPORTED_PLATFORMS: ${{ convertToJson(parameters.supportedPlatforms) }}

- pwsh: |
Import-Module $Env:SYSTEM_DEFAULTWORKINGDIRECTORY/xamarin-macios/tools/devops/automation/scripts/MaciosCI.psd1
gci env: | sort | format-table -autosize -wrap
$apiScanMatrix = Get-APIScanConfiguration -EnabledPlatforms "$Env:CONFIGURE_PLATFORMS_DOTNET_PLATFORMS"
# compress the json to remove any newlines, because we can't set the variable below if the json has any newlines
Write-Host "$apiScanMatrix"
$apiScanMatrix = $apiScanMatrix | ConvertFrom-Json | ConvertTo-Json -Compress
Write-Host "##vso[task.setvariable variable=APISCAN_MATRIX;isOutput=true]$apiScanMatrix"
name: apiscan_matrix
displayName: 'Create APIScan matrix'

# upload config to be consumed later
- ${{ if eq(parameters.uploadArtifacts, true) }}:
- task: PublishPipelineArtifact@1
Expand Down
28 changes: 28 additions & 0 deletions tools/devops/automation/templates/governance/apiscan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
parameters:

- name: isPR
type: boolean

- name: repositoryAlias
type: string
default: self

- name: commit
type: string
default: HEAD

- name: uploadPrefix
type: string
default: '$(MaciosUploadPrefix)'

steps:

- template: ../common/checkout.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}

- pwsh: |
Write-Output "Performing APISCan for $(PLATFORM) using tenant $(TENANT) client id $(CLIENT_ID) and secret $(SECRET)."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the secret be displayed like this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devops is super smart (for once), and knows not to display them:

Screenshot 2024-02-02 at 15 29 11

displayName: "APISCan"
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ parameters:

steps:

- template: ./common/checkout.yml
- template: ../common/checkout.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
Expand All @@ -31,7 +31,6 @@ steps:
Dir $(System.DefaultWorkingDirectory)
displayName: Show directories


- powershell: |
Get-ChildItem -Recurse -Path $(Build.SourcesDirectory)/maccore
displayName: Show maccore content
Expand Down
69 changes: 69 additions & 0 deletions tools/devops/automation/templates/governance/stage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# contains the stage used to run all governance related jobs

parameters:
- name: isPR
type: boolean

- name: repositoryAlias
type: string
default: self

- name: commit
type: string
default: HEAD

- name: stageDisplayNamePrefix
type: string
default: ''


stages:
- stage: governance_checks
displayName: '${{ parameters.stageDisplayNamePrefix }}Governance Checks'
dependsOn: [ configure_build ]

jobs:
- job: apiscan
displayName: 'APIScan:'
pool:
vmImage: windows-latest


strategy:
matrix: $[ stageDependencies.configure_build.configure.outputs['apiscan_matrix.APISCAN_MATRIX'] ]

steps:
- template: ./apiscan.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}

- job: general_governance
displayName: 'Governance Checks'
pool:
vmImage: windows-latest

steps:
- template: ./general.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}

- job: tsa_upload
displayName: 'TSA Upload'
dependsOn: [ general_governance, apiscan ]
pool:
vmImage: windows-latest

variables:
${{ each p in parameters.platforms }}:
INCLUDE_DOTNET_${{ upper(p.key) }}: $[ stageDependencies.configure_build.configure.outputs['configure_platforms.INCLUDE_DOTNET_${{ upper(p.key)}}'] ]

steps:
- template: ./tsa-upload.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}
28 changes: 28 additions & 0 deletions tools/devops/automation/templates/governance/tsa-upload.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
parameters:

- name: isPR
type: boolean

- name: repositoryAlias
type: string
default: self

- name: commit
type: string
default: HEAD

- name: uploadPrefix
type: string
default: '$(MaciosUploadPrefix)'

steps:

- template: ../common/checkout.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}

- pwsh: |
Write-Output "Upload to tsa."
displayName: "Debug"
23 changes: 6 additions & 17 deletions tools/devops/automation/templates/main-stage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -277,23 +277,12 @@ parameters:
stages:

- ${{ if eq(parameters.runGovernanceTests, true) }}:
- stage: governance_checks
displayName: '${{ parameters.stageDisplayNamePrefix }}Governance Checks'
dependsOn: build_packages
${{ if and(ne(parameters.dependsOn, ''), ne(parameters.dependsOnResult, '')) }}:
condition: eq(dependencies.${{ parameters.dependsOn }}.result, '${{ parameters.dependsOnResult }}')
jobs:
- job: governance
displayName: 'Governance Checks'
pool:
vmImage: windows-latest
steps:
- template: governance-checks.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}

- template: ./governance/stage.yml
parameters:
isPR: ${{ parameters.isPR }}
repositoryAlias: ${{ parameters.repositoryAlias }}
commit: ${{ parameters.commit }}
stageDisplayNamePrefix: ${{ parameters.stageDisplayNamePrefix }}

- ${{ if parameters.isPR }}:
- stage: clean
Expand Down
Loading