Move bicep modules into registry

[rajeshkamal5050]

Bicep has the ability to reference modules that are contained within an Azure Container Registry.

That registry can be public or private.

For azd, we'll want our modules stored in a public registry so our end users don't need to authenticate to that registry.

For that we have two options:

1. Build our own
2. Use the Bicep public registry https://github.com/Azure/bicep-registry-modules

I would like us to address this in phases:

Phase 1:

1. Create an Azure Container Registry to host our modules
2. Version and push those modules to the registry like we do with templates
3. Update template bicep files to point to the modules in the registry instead of local - and remove /bicep/core from templates.
4. Provide an easy way for customers to eject from use the registry modules to local source. I think that is best done manually as a first step. We can direct the users to the correct path in our azure-dev repo and tell them how to copy them, where to place them, etc. Other than the references to local files versus registry, there should be no changes required from them to the actual module code.

Phase 1 Tasks

• Decide on a versioning scheme for bicep registry modules, likely semver
• Integrate bicep registry module publishing into our process
• Figure out how to update module version references in templates when they change - i.e. bicep module changes in a PR, we want that PR to publish those modules and update the template to point to them. Then when we ship, we want the resulting templates to use that version
• Decide where we'll host our modules. Should we use the same ACR that we use for containers? The downside to doing that is we'll likely need a "bicep" folder in the path
• Figure out an eject plan. Right now we have a comment at the top of the bicep file telling people where to find the source.

Phase 2:

1. Once we have Phase 2, we discuss moving azd modules to the public bicep registry
2. Change our templates to use the modules in the public registry.
3. Figure out a better eject plan. Potentially having a command that downloads all of teh files from the registry to the local machine.

For us to do this I would see us needing:

1. Buy-in from Azure service teams to maintain their bicep modules or a team staffed to support the creating and maintaining of the modules
2. Method for upgrading our templates to latest version of the public modules and a way to run CI against those versions.

Additional Info

Azure has individual modules here: https://github.com/azure/resourcemodules. Once bicep releases typing and the resourcemodules move to typing for params, and they are pushed to a public repo, then we should evaluate moving to those as well.

Tasks

Preview Give feedback

1. Design IaC module versioning scheme and breaking change policy #1731
   Bicep feature +2

[savannahostrowski]

We should also think about what inner loop development looks like when we move Bicep modules to a registry, using VS Code as our hero client. The primary concern here is that after we move them into a registry, the local development experience becomes a little magical (as there won't be files for them to code nav to).

This would likely and ideally fall into the Bicep extension's feature scope but we should work with their team to ensure that we have an answer for this.

[jongio]

I have been experimenting with putting /infra/core in a public bicep registry and have documented the plan here: #751

With this update, we don't need to ship infra/core files to every template.  We instead reference the modules that are in a public registry (that we own for now).

You can see what the resulting template will look like here: https://github.com/Azure-Samples/todo-nodejs-mongo/tree/pr/1251

You'll notice that the infra folder no longer has a core folder:

All of the core modules are now referenced with the "br" aka "bicep registry" notation

Could you please have a look and let me know your thoughts on this approach?

Jon

[vhvb1989]

Phase 1 (using ACR, pushing core modules, updating none-core modules) , PR: #1267

[puicchan]

Related Bicep issues:
Completions

• Set up CDN for MCR module reference completion - Add new endpoint for MCR public module reference completion bicep#9036
• Support MCR module reference completion Support MCR module reference completions bicep#9037
• Support ACR module reference completion Support ACR (private) module reference completions bicep#9038 - will have partial support. This has dependency on ACR.

Help/hint/F12

• Add view Type VS Code - “View Type Documentation” to click and open readme for module in Bicep registry  bicep#8789
• Add help text Provide a help text when consuming modules from ACR bicep#6356
• F12 (go to definition) Story: Go to Definition "source map" support for Container Registry modules bicep#5407

[danieljurek]

From an Engineering System perspective, I think (this is a brain dump... plans that make sense coming soon):

• Bicep modules look like "independent" shippable units compared to the azd CLI and our Azure-Samples templates. To that end we should have a separate pipeline which publishes those.

  • Modules should also be independent of each other. That is, we can release br/azd:bicep/staging/core/host/appservice:v1.0 and br/azd:bicep/staging/core/host/cosmos-mongo-db:v2.0 in the same or separate pipeline runs
  • Modules live in a separate folder from repoman's templates folder in the repo (unless we want to add this directly to repoman scenarios)

• Any release process should involve checking for presence of a module at a version before releasing OVER that version... Maybe this can work with --write-enabled false

• Use SemVer to version the modules, this is primarily for the benefit of consumers though upgrading on the consumer side looks to be manual.

• We'll need a way to test PR-specific and latest modules against templates

Dev/PR/Release process for azd developers working on bicep modules looks like:

1. Create project-specific ACR instance
2. Update bicepconfig.json to point to project-specific ACR instance, bicep push modules in their current state to ACR
3. Make changes, bicep push to project-specific ACR instance as needed, ensure versions of releasing modules are correct
4. Open PR, run azure-dev - bicep-modules and azure-dev - template - tests with updated bicep modules included (those pipelines have a PR configuration which uses the proposed config changes)
5. Get approval and merge
6. Kick off manual run of azure-dev - bicep-modules pipeline with the names of the modules to release
7. Approve release of modules
8. Kick off manual run of azure-dev - repoman and approve template release

This workflow incorporates both template and bicep module changes in the same PR. It's more complex but it's also more complete. If we broke this into two PRs (one to update the bicep modules and one to update the template references to the bicep modules) that would also work and would probably require less automation.

We'll also have to figure out how to ensure that the templates point only to intended released bicep module versions.

Keeping templates and published bicep modules in sync will also require some design. We can publish new modules at any time but templates may be held up for other reasons.

[jongio]

@danieljurek - thx for that.

I think we could consider shipping azd bicep core modules as a versioned unit versus versioning each module independently. My hunch is that versioning individually would create more of a maintenance issue than worth.

[rajeshkamal5050]

@vhvb1989 in your POC. Did you try referencing modules and running azd from an external subscription or personal account? Will we run into any access issues since being a private registry?

[vhvb1989]

@rajeshkamal5050 , yes, I did try. The container registry needs to be configured to enable anonymous read access. That's anyone can pull the modules w/o any credentials. The registry behaves similar to a CDN.
Docs: https://learn.microsoft.com/en-us/azure/container-registry/anonymous-pull-access#about-anonymous-pull-access

[rajeshkamal5050]

@savannahostrowski @ellismg Not in scope for GA. Moving this to Gallium.

[vhvb1989]

related: #1267

[savannahostrowski]

Assigning to Germanium for now - need to confirm with other stakeholders on what's necessary here.

[vhvb1989]

@rajeshkamal5050 @savannahostrowski do you still want to see this happening? or should we close it in favor of [AVMs, ADE, IaC abstraction, etc, etc], as the strategy for composing infra, instead of introducing this azd-bicep-modules-registry

[savannahostrowski]

Yeah - I'm actually okay with that for now.