Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Modules] Add Synapse Workspace module #928

Merged
merged 87 commits into from
Jul 20, 2022

Conversation

renepajta
Copy link
Member

@renepajta renepajta commented Jan 25, 2022

Change

Add support for deploying Synapse Workspace. The PR implements basic configuration and more advanced configuration will come in future PRs.

Pipeline reference

Build Status

Dependencies

  • ADLS Gen 2 storage account and file system
  • Azure KeyVault and Encryption key for enabling encryption

Type of Change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I'm sure there are no other open Pull Requests for the same update/change
  • My corresponding pipelines / checks run clean and green without any errors or warnings
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (readme)
  • I did format my code

renepajta and others added 2 commits January 20, 2022 11:35
* Added support for Synapse Private Link Hubs

* Fixed typo in Github workflow

* Create new workflow for deploying Synapse Private Link Hub

* Added support for Locks, RBAC and Private Endpoints

* Updated ReadMe file

* Fixed outputs & updated ReadMe

* Switched back to main branch

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/.bicep/nested_rbac.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/.bicep/nested_rbac.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <[email protected]>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <[email protected]>

* Fixed naming, added ADO pipeline, updated README

* Removed TODOs in ReadMe

* Reversed changes in auto-generated Readme to avoid conflicts

* Remove lock settings from parameter file

* Updated based on comments

* Updated readme.md

* Initial commit for Synapse Workspace module

* Fixed failed Pester tests

* Fixes for failed Pester tests

* Fix: Workspace cannot be created with only UserAssigned identity.

* The default data lake storage account url property should be in the format: https://accountname.dfs.core.windows.net

* Added Diagnostics settings and additional set of parameters

* Add param file to pipeline

* Updated readme file

* Added test cases for encryption configuration

* Added test case for managed vnet

* Updated workspace name to be unique

* Removed unsupported setting - AzureADOnlyAuth

* Added ADO pipeline for Synapse Workspace module

* Created unique names

* Updated resource names, switched from name to resource id for storage account

* Parameter file should not contain the Subscription ID guid

* Changed from resource id to name for default storage account, provided tenant id instead of tenant name

* Updated param reference to Key Vault keys

* Fixed diagnostics account id

* Switched GitHub Workflow to trigger on main branch

* Removed fetch from upstream repo

* Switched back to trigger on current branch for testing

* Switched back to trigger on current branch for testing

* Added missing UAI to parameters file

* Updated process for MSI encryption

* Fixed Pester errors

* Fixed deployment errors

* Deployment fixes

* Deployment fixes

* Deployment fixes

* Deployment fixes

* Deployment fixes

* Slim down the tests

* Deployment fixes

* Deployment fixes

* Deployment fixes 1

* Deployment fixes 2

* Removed testing deployment configuration

* Removed unused code blocks

* Fixed Linter issues

* Updated README

* Added support for Encryption Key Vault with Access Policies

* Trigger on branch to test before merge

* Trigger GH action

* Added all validation steps

* Updated README

* Trigger workflow

* Removed changes

* Revert "Removed changes"

This reverts commit 8cf78a0.

* Revert "Added all validation steps"

This reverts commit 8f95b10.

* Revert "Updated README"

This reverts commit a0b4c49.

* Fixing weird issues

* Updated messsages from Test function

* Fixed missing if condition

* Removed scope fro RBAC assignment

* Removed table formatting

* Added missing parent to access policy

* Resolved code formatting issues

* Updated README

* Moved to dependency keyvault

Co-authored-by: Alexander Sehr <[email protected]>
@AlexanderSehr AlexanderSehr added [cat] modules category: modules [cat] pipelines category: pipelines labels Jan 25, 2022
@AlexanderSehr AlexanderSehr added this to the v 0.4 milestone Jan 25, 2022
@AlexanderSehr AlexanderSehr linked an issue Jan 25, 2022 that may be closed by this pull request
@github-actions
Copy link

github-actions bot commented Jan 25, 2022

Unit Test Results

  1 files  ±    0    1 suites  ±0   15s ⏱️ -27s
11 tests  -   19  11 ✔️  - 19    0 💤 ±    0  0 ±0 
76 runs   - 256  11 ✔️  - 19  65 💤  - 237  0 ±0 

Results for commit c65cedf. ± Comparison against base commit 80087b3.

This pull request removes 30 and adds 11 tests. Note that renamed tests count towards both.
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/authorizationRules] used resource type [namespaces/AuthorizationRules] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/disasterRecoveryConfigs] used resource type [namespaces/disasterRecoveryConfigs] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/ipFilterRules] used resource type [namespaces/ipfilterrules] should use one of the recent API version(s). Currently using [2018-01-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/migrationConfigurations] used resource type [namespaces/migrationConfigurations] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues/authorizationRules] used resource type [namespaces/queues/authorizationRules] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [locks] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [namespaces/queues/authorizationRules] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [namespaces/queues] should use one of the recent API version(s). Currently using [2021-06-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [roleassignments] should use one of the recent API version(s). Currently using [2021-04-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/topics/authorizationRules] used resource type [namespaces/topics/authorizationRules] should use one of the recent API version(s). Currently using [2021-06-01-preview]
…
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces/keys] used resource type [diagnosticsettings] should use one of the recent API version(s). Currently using [2021-05-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces/keys] used resource type [workspaces/keys] should use one of the recent API version(s). Currently using [2021-06-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [diagnosticsettings] should use one of the recent API version(s). Currently using [2021-05-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [locks] should use one of the recent API version(s). Currently using [2016-09-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [privateEndpoints/privateDnsZoneGroups] should use one of the recent API version(s). Currently using [2021-05-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [privateEndpoints] should use one of the recent API version(s). Currently using [2021-05-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [roleassignments] should use one of the recent API version(s). Currently using [2020-04-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [roleassignments] should use one of the recent API version(s). Currently using [2021-04-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [vaults/accessPolicies] should use one of the recent API version(s). Currently using [2021-06-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [workspaces/keys] should use one of the recent API version(s). Currently using [2021-06-01]
…

♻️ This comment has been updated with latest results.

@AlexanderSehr AlexanderSehr requested a review from eriqua July 18, 2022 12:09
@AlexanderSehr AlexanderSehr requested a review from eriqua July 18, 2022 17:26
Copy link
Contributor

@eriqua eriqua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

@eriqua eriqua dismissed stale reviews from AlexanderSehr and MariusStorhaug July 20, 2022 19:03

All comments addressed

@AlexanderSehr AlexanderSehr merged commit 2a3a325 into main Jul 20, 2022
@AlexanderSehr AlexanderSehr deleted the users/repajta/852_synapse_workspace_module branch July 20, 2022 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[cat] modules category: modules [cat] pipelines category: pipelines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement template for Synapse Workspace
5 participants