Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use repositories file by default for GH scripts #8527

Merged
merged 3 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 78 additions & 71 deletions tools/github/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Overview

This area is focused on the management of the Azure SDK GitHub repositories, containing tools and data used for that purpose.
This area is focused on the management of the Azure SDK GitHub repositories, containing tools and data used for that purpose.

## Prerequisites

Expand All @@ -10,83 +10,90 @@ This area is focused on the management of the Azure SDK GitHub repositories, con

## Structure

* **root**
jsquire marked this conversation as resolved.
Show resolved Hide resolved
- **root**

_The root contains the scripts used for repository management and their associated data._

* **scripts**
- **scripts**

_The core scripts for repository management, such as managing milestones and labels._

* **data**
- **data**

_The data associated with the scripts. This includes items such as the set of centrally managed Azure SDK repositories and the labels common to all repositories._

* **data/repository-snapshots**
- **data/repository-snapshots**

_The container for a snapshot of the labels that exist in each managed repository; primarily used to help detect and report on new labels that have been added outside the common set._

## Scripts

### `Add-AzsdkMilestones.ps1`

_Creates a set of milestones in one or more repositories. The milestones follow Azure SDK conventions for naming and due dates. With the default parameter set, milestones will be created for the next 6 months in repositories in the 'repositories.txt' file in the script directory._

```powershell
# Create the default set of milestone and repository data.
./Add-AzsdkMilestones.ps1

# Create milestones in the .NET and Java repositories for a specific date range.
./Add-AzsdkMilestones.ps1 -Languages 'net', 'java' -StartDate 2023-01-01 -EndDate 2023-07-31

# View the help for the full set of parameters.
get-help ./Add-AzsdkMilestones.ps1 -full
```

### `Add-AzsdkProjectIssues.ps1`
_Adds a set of Azure SDK repository issues tagged with a given set of labels to a GitHub project. With the default parameter set, the issues with the specified labels will be queried from a set of core language repositories._

```powershell
# Adds issues with the "Client" and "KeyVault" labels to project #150, querying a set
# of core language repositories.
./Add-AzsdkProjectIssues.ps1 -ProjectNumber 150 -Labels Client, KeyVault

# Adds issues with the "KeyVault" label in the .NET repository to project #150, setting
# the "Status" custom field to "Todo".
./Add-AzsdkProjectIssues.ps1 -Languages 'net' -ProjectNumber 150 -Labels KeyVault -Fields @{Status="Todo"}

# View the help for the full set of parameters.
get-help ./Add-AzsdkProjectIssues.ps1 -full
```

### `Sync-AzsdkLabels.ps1`
_Creates or updates the set of labels expected to be common across the Azure SDK repositories, ensuring that names, descriptions, and colors share the common configuration._

```powershell
# Uses the files from the `data` directory to synchronize the common labels to all centrally managed
# repositories using the default delay between each repository to guard against GitHub throttling.
./Sync-AzsdkLabels.ps1

# Synchronize the common labels to the Azure SDK for .NET repository.
./Sync-AzsdkLabels.ps1 -LabelsFilePath "../data/common-labels.csv" -Languages 'net'

# View the help for the full set of parameters.
get-help ./Sync-AzsdkLabels.ps1 -full
```

### `Snapshot-AzsdkLabels.ps1`
_Creates or updates the set of labels expected to be common across the Azure SDK repositories, ensuring that names, descriptions, and colors share the common configuration._

```powershell
# Uses the files from the "data" directory to generate snapshots of non-common labels for each
# of the centrally managed repositories in the "data/repository-snapshots" directory while
# writing any new non-common labels created for the repository to the host.
./Snapshot-AzsdkLabels.ps1 -Diff

# Create a snapshot for the Azure SDK for .NET repository, treating the labels defined in
# "../data/common-labels.csv" as the expected common set. The resulting snapshot is written to the ""./snapshots" directory.
./Snapshot-AzsdkLabels.ps1 -LabelsFilePath "../data/common-labels.csv" -Languages 'net' -RepositoryFilePath "snapshots"

# View the help for the full set of parameters.
get-help ./Snapshot-AzsdkLabels.ps1 -full
```

## Scripts

### `Add-AzsdkMilestones.ps1`

_Creates a set of milestones in one or more repositories. The milestones follow Azure SDK conventions for naming and due dates. With the default parameter set, milestones will be created for the next 6 months in repositories in the 'repositories.txt' file in the script directory._

```powershell
# Create the default set of milestone and repository data.
./Add-AzsdkMilestones.ps1

# Create milestones in the .NET and Java repositories for a specific date range.
./Add-AzsdkMilestones.ps1 -Languages 'net', 'java' -StartDate 2023-01-01 -EndDate 2023-07-31

# View the help for the full set of parameters.
get-help ./Add-AzsdkMilestones.ps1 -full
```

### `Add-AzsdkProjectIssues.ps1`

_Adds a set of Azure SDK repository issues tagged with a given set of labels to a GitHub project. With the default parameter set, the issues with the specified labels will be queried from a set of core language repositories._

```powershell
# Adds issues with the "Client" and "KeyVault" labels to project #150, querying a set
# of core language repositories.
./Add-AzsdkProjectIssues.ps1 -ProjectNumber 150 -Labels Client, KeyVault

# Adds issues with the "KeyVault" label in the .NET repository to project #150, setting
# the "Status" custom field to "Todo".
./Add-AzsdkProjectIssues.ps1 -Languages 'net' -ProjectNumber 150 -Labels KeyVault -Fields @{Status="Todo"}

# View the help for the full set of parameters.
get-help ./Add-AzsdkProjectIssues.ps1 -full
```

### `Sync-AzsdkLabels.ps1`

_Creates or updates the set of labels expected to be common across the Azure SDK repositories, ensuring that names, descriptions, and colors share the common configuration._

```powershell
# Uses the files from the `data` directory to synchronize the common labels to all centrally managed
# repositories using the default delay between each repository to guard against GitHub throttling.
./Sync-AzsdkLabels.ps1

# Synchronize the common labels to the Azure SDK for .NET repository.
./Sync-AzsdkLabels.ps1 -LabelsFilePath "../data/common-labels.csv" -Languages 'net'

# View the help for the full set of parameters.
get-help ./Sync-AzsdkLabels.ps1 -full
```

### `Snapshot-AzsdkLabels.ps1`

_Creates or updates the set of labels expected to be common across the Azure SDK repositories, ensuring that names, descriptions, and colors share the common configuration._

```powershell
# Uses the files from the "data" directory to generate snapshots of non-common labels for each
# of the centrally managed repositories in the "data/repository-snapshots" directory while
# writing any new non-common labels created for the repository to the host.
./Snapshot-AzsdkLabels.ps1 -Diff

# Create a snapshot for the Azure SDK for .NET repository, treating the labels defined in
# "../data/common-labels.csv" as the expected common set. The resulting snapshot is written to the ""./snapshots" directory.
./Snapshot-AzsdkLabels.ps1 -LabelsFilePath "../data/common-labels.csv" -Languages 'net' -RepositoryFilePath "snapshots"

# View the help for the full set of parameters.
get-help ./Snapshot-AzsdkLabels.ps1 -full
```

## References and Resources

- [GitHub CLI Docs](https://docs.github.com/en/github-cli)
21 changes: 5 additions & 16 deletions tools/github/scripts/Add-AzsdkMilestones.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,11 @@ param (
[ValidateScript({Test-Path $_ -PathType 'Leaf'})]
[string]$RepositoryFilePath = "$PSScriptRoot/../data/repositories.txt",

[Parameter(ParameterSetName = 'Repositories')]
[ValidateNotNullOrEmpty()]
[string[]] $Repositories = @(
'Azure/azure-sdk-for-cpp'
'Azure/azure-sdk-for-go'
'Azure/azure-sdk-for-java'
'Azure/azure-sdk-for-js'
'Azure/azure-sdk-for-net'
'Azure/azure-sdk-for-python'
'Azure/azure-sdk-for-rust'
'Azure/azure-sdk-tools'
),

[Parameter(ParameterSetName = 'Languages')]
[ValidateNotNullOrEmpty()]
[string[]] $Languages = @('cpp', 'go', 'java', 'js', 'net', 'python', 'rust', 'c', 'ios', 'android'),
[Parameter(ParameterSetName = 'Repositories', Mandatory = $true)]
[string[]] $Repositories,

[Parameter(ParameterSetName = 'Languages', Mandatory = $true)]
[string[]] $Languages,

[Parameter()]
[DateTimeOffset] $StartDate = [DateTimeOffset]::Now,
Expand Down
24 changes: 6 additions & 18 deletions tools/github/scripts/Add-AzsdkProjectIssues.ps1
Original file line number Diff line number Diff line change
@@ -1,28 +1,16 @@
[CmdletBinding(DefaultParameterSetName = 'Repositories', SupportsShouldProcess = $true)]
[CmdletBinding(DefaultParameterSetName = 'RepositoryFile', SupportsShouldProcess = $true)]
param (
[Parameter(Mandatory = $true, Position = 0)]
[int] $ProjectNumber,

[Parameter(Mandatory = $true, Position = 1)]
[string[]] $Labels,

[Parameter(ParameterSetName = 'Repositories')]
[Alias("repos")]
[ValidateNotNullOrEmpty()]
[string[]] $Repositories = @(
'Azure/azure-sdk-for-cpp'
'Azure/azure-sdk-for-go'
'Azure/azure-sdk-for-java'
'Azure/azure-sdk-for-js'
'Azure/azure-sdk-for-net'
'Azure/azure-sdk-for-python'
'Azure/azure-sdk-for-rust'
'Azure/azure-sdk-tools'
),

[Parameter(ParameterSetName = 'Languages')]
[ValidateNotNullOrEmpty()]
[string[]] $Languages = @('cpp', 'go', 'java', 'js', 'net', 'python', 'rust', 'c', 'ios', 'android'),
[Parameter(ParameterSetName = 'Repositories', Mandatory = $true)]
[string[]] $Repositories,

[Parameter(ParameterSetName = 'Languages', Mandatory = $true)]
[string[]] $Languages,

[Parameter(ParameterSetName = 'RepositoryFile')]
[ValidateScript({Test-Path $_ -PathType 'Leaf'})]
Expand Down
29 changes: 9 additions & 20 deletions tools/github/scripts/Snapshot-AzsdkLabels.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,11 @@ param (
[ValidateScript({Test-Path $_ -PathType 'Leaf'})]
[string] $RepositoryFilePath = "$PSScriptRoot/../data/repositories.txt",

[Parameter(ParameterSetName = 'Repositories')]
[ValidateNotNullOrEmpty()]
[string[]] $Repositories = @(
'Azure/azure-sdk-for-cpp'
'Azure/azure-sdk-for-go'
'Azure/azure-sdk-for-java'
'Azure/azure-sdk-for-js'
'Azure/azure-sdk-for-net'
'Azure/azure-sdk-for-python'
'Azure/azure-sdk-for-rust'
'Azure/azure-sdk-tools'
),

[Parameter(ParameterSetName = 'Languages')]
[ValidateNotNullOrEmpty()]
[string[]] $Languages = @('cpp', 'go', 'java', 'js', 'net', 'python', 'rust', 'c', 'ios', 'android'),
[Parameter(ParameterSetName = 'Repositories', Mandatory = $true)]
[string[]] $Repositories,

[Parameter(ParameterSetName = 'Languages', Mandatory = $true)]
[string[]] $Languages,

[Parameter()]
[ValidateRange(0,100)]
Expand Down Expand Up @@ -76,27 +65,27 @@ foreach ($repo in $Repositories) {
}

$filePath = (Join-Path $SnapshotDirectory "$($repo.Substring(($repo.IndexOf('/') + 1))).txt")
$snapshot = ((ConvertFrom-Json $repositoryLabels)| where { -not $commonLabels.Contains($_.Name) }).Name
$snapshot = ((ConvertFrom-Json $repositoryLabels)| Where-Object { -not $commonLabels.Contains($_.Name) }).Name

if ($Diff) {
$previousSnapshot = [Collections.Generic.HashSet[string]]::new()

if (Test-Path $filePath) {
(Get-Content $filePath) | select { $previousSnapshot.Add($_) } | Out-Null
(Get-Content $filePath) | Select-Object { $previousSnapshot.Add($_) } | Out-Null
}

Write-Host "==================================================== " -ForegroundColor Green
Write-Host " $($repo)" -ForegroundColor Green
Write-Host "==================================================== " -ForegroundColor Green
Write-Host "New labels added since the last snapshot:"

$newLabels = $snapshot | where { -not $previousSnapshot.Contains($_) }
$newLabels = $snapshot | Where-Object { -not $previousSnapshot.Contains($_) }

if ($newLabels.Count -eq 0) {
Write-Host "`t None"
}
else {
$newLabels | foreach { Write-Host "`t$_" }
$newLabels | ForEach-Object { Write-Host "`t$_" }
}

Write-Host ""
Expand Down
21 changes: 5 additions & 16 deletions tools/github/scripts/Sync-AzsdkLabels.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,11 @@ param (
[ValidateScript({Test-Path $_ -PathType 'Leaf'})]
[string] $RepositoryFilePath = "$PSScriptRoot/../data/repositories.txt",

[Parameter(ParameterSetName = 'Repositories')]
[ValidateNotNullOrEmpty()]
[string[]] $Repositories = @(
'Azure/azure-sdk-for-cpp'
'Azure/azure-sdk-for-go'
'Azure/azure-sdk-for-java'
'Azure/azure-sdk-for-js'
'Azure/azure-sdk-for-net'
'Azure/azure-sdk-for-python'
'Azure/azure-sdk-for-rust'
'Azure/azure-sdk-tools'
),

[Parameter(ParameterSetName = 'Languages')]
[ValidateNotNullOrEmpty()]
[string[]] $Languages = @('cpp', 'go', 'java', 'js', 'net', 'python', 'rust', 'c', 'ios', 'android'),
[Parameter(ParameterSetName = 'Repositories', Mandatory = $true)]
[string[]] $Repositories,

[Parameter(ParameterSetName = 'Languages', Mandatory = $true)]
[string[]] $Languages,

[Parameter()]
[ValidateRange(0,100)]
Expand Down
10 changes: 2 additions & 8 deletions tools/github/scripts/Validate-AzsdkCodeOwner.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,15 @@ $hasPermissions = $false
# Verify that the user exists and has the correct public
# organization memberships.
$orgResponse = (gh api "https://api.github.com/users/$UserName/orgs")
$orgs = $orgResponse | ConvertFrom-Json

if ($orgs -ne $null) {
$orgs = $orgs | select -Expand login
} else {
$orgs = @()
}
$orgs = $orgResponse | ConvertFrom-Json | Select-Object -Expand login -ErrorAction Ignore

# Validate that the user has the required public organization memberships.
$requiredOrgs = [System.Collections.Generic.HashSet[String]]::new([StringComparer]::InvariantCultureIgnoreCase)
$requiredOrgs.Add("Microsoft") | Out-Null
$requiredOrgs.Add("Azure") | Out-Null

# Capture non-required organizations for verbose output.
$otherOrgs = $orgs | where { -not $requiredOrgs.Contains($_) }
$otherOrgs = $orgs | Where-Object { -not $requiredOrgs.Contains($_) }

Write-Host ""
Write-Host "Required Orginizations:" -ForegroundColor DarkGray
Expand Down