[ISSUE] Issue with Unity Catalog resources that are on Workspace API

[zshao9]

I have a list of Unity Catalogs (one for each region) whose catalog/schemas/grants need to be created with Terraform.

Since those resources need to go to the Workspace-API providers, I need a list of providers one for each workspace (associated with the Unity Catalog).

However, Terraform doesn't support a dynamic list of providers (for for_each). See:

• 2019 issue: Instantiating Multiple Providers with a loop hashicorp/terraform#19932
• 2020 issue Ability to pass providers to modules in for_each hashicorp/terraform#24476
• 2022 discussions: https://stackoverflow.com/questions/72541598/using-for-each-with-provider-setting

Can we consider to use "metastore_id" properly on the catalog/schema/grants resources?
Then I just need to use a single provider with whatever workspace for all the different metastores.

[alexott]

No, it's not possible because such functionality isn't available in the UC APIs.

[mgyucht]

Just to expand on this a bit: workspace-level UC APIs only allow interacting with the metastore assigned to that workspace. Even if the TF provider were to accept a metastore ID, it wouldn't be able to interact with that metastore without a running workspace that is assigned to said metastore. We're working with the UC team to align on exposing these APIs at the account-level, in which case you would be able to interact with them purely when configured at the account level.

[bobdoah]

This would be incredibly useful

[sreedhanyagn]

We have more than 100 workspaces our organization and defining provider for each workspace is not feasible. Is there any alternative solution for this ?
Can we use a single workspace as provider and create Catalogs for other workspaces using this provider ?

[zshao9]

As long as they share the same unity catalog (e.g. in the same region) then this is possible. Curious what is the use case of 100+ workspaces? One per user?

…

On Sat, Jan 27, 2024 at 1:11 PM Sree ***@***.***> wrote: We have more than 100 workspaces our organization and defining provider for each workspace is not feasible. Is there any alternative solution for this ? Can we use a single workspace as provider and create Catalogs for other workspaces using this provider ? — Reply to this email directly, view it on GitHub <#3018 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BD7W6HSD2VSXXUSSLUXOCUTYQVUQXAVCNFSM6AAAAABAPDFMZSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTGMZTOMBQGI> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[sreedhanyagn]

Hey @zshao9 , thanks for your reply.
We are supporting many teams and each has team has multiple workspaces in different environments.

[zshao9]

Got it. Are they all in the same region? If so, then they use the same unity catalog. You can just set up an "admin workspace" where all these Terraform operations are done.

…

On Sat, Jan 27, 2024 at 2:19 PM Sree ***@***.***> wrote: Hey @zshao9 <https://github.com/zshao9> , thanks for your reply. We are supporting many teams and each has team has multiple workspaces in different environments. — Reply to this email directly, view it on GitHub <#3018 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BD7W6HR4NK4MSDP5KU5YE5DYQV4N5AVCNFSM6AAAAABAPDFMZSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTGM2TEMJWGI> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[sreedhanyagn]

Yes, they all are in the same region.