Allow LINQ-style projections from arrays

[WhitWaldo]

Is your feature request related to a problem? Please describe.
I have to deploy several public IP addresses as part of my deployment. I maintain an array variable in my deployment template that lists out this IP address information and all manner of other values (typically because IP addresses correspond with other resources deployed to a number of also-specified subnets). For example, this might look something like this:

var subnets = [
  {
    name: 'DataExplorer'
    cidr: '10.0.12.0/24'
    hasLoadBalancer: false
    isPublicLoadBalancer: false
    isPublicIp: false
    isStandardIpSku: true
    isStaticIp: false
    ipZones: []
    enablePrivateLink: true
    hasRouteTable: true
    nsgName: DataExplorerNsgName
    nsgRules: []
    delegations: [
      {
        name: 'AzureDataExplorer'
        properties: {
          serviceName: 'Microsoft.Kusto/clusters'
        }
      }
    ]
    serviceEndpoints: []
  }
  {
    name: 'ApplicationGateway'
    cidr: '10.0.0.0/21'
    hasLoadBalancer: false
    isPublicLoadBalancer: false
    isPublicIp: true
    isStandardIpSku: true
    isStaticIp: true
    ipZones: [
      '1'
      '2'
      '3'
    ]
    enablePrivateLink: false
    hasRouteTable: false
    nsgName: ''
    nsgRules: []
    delegations: []
    serviceEndpoints: []
  }
]

I have a great many subnets, but you get the point. It's a central configuration object for both how the subnet is configured, but also general networking details for services that use this particular section. Because it's an array, I can then easily use it to spin up public IP addresses via another module (not detailed here):

var pipNames = [for (subnet, index) in subnets: {
  name: '${DeploymentPrefix}-${subnet.name}-${resourceMapping['publicIp']}'
}]
module PIP './resources/publicIpAddress.bicep' = [for (subnet, index) in subnets: if (subnet.isPublicIp) { //Don't necessarily need to provision an IP address for every subnet
  name: '${resourceMapping['deployment']}-${DeploymentPrefix}-${subnet.name}-${resourceMapping['publicIp']}'
  params: {
    IpZones: subnet.ipZones
    AllocationMethod: subnet.isStaticIp ? 'Static' : 'Dynamic'
    IsStandardSku: subnet.isStandardIpSku
    IpAddressName: pipNames[index].name
    Location: Location
  }
}]

All this works without issue - where having a projection, particularly a filter-able one would be useful is later on when I'm actually creating say, an application gateway. Say I need to actually know the name of the public IP address resource that was created earlier. Today this looks like this (knowing that the application gateway subnet is the first element in the array):

var appGwName = 'appGw01'
module AppGw 'appgw.bicep' = {
  name: appGwName
  params: {
    PublicIpName: pipNames[1].name //Note we created the array of names earlier based on the subnet values - since it's subnet[1], it must also be pipNames[1]
  }
}

The problem isn't in setting this up the first time - it's in maintaining it. Say I don't need the DataExplorer subnet any longer, so I remove the subnet object from the array. This is now broken and I won't know it until I attempt to deploy and either the index is out of range (or worse, it deploys with the wrong information and I don't catch it until something doesn't work much further downstream).

This is a huge maintainability problem.

Describe the solution you'd like
I would like to be able to apply a filterable projection against an array at run-time (bonus points if the Intellisense can work out what I'm doing and provide rich typing support) not unlike what I can do with LINQ statements in C#.

Assuming I can only use those values that are known ahead of time (as is generally true today), I'd like to be able to apply a filter to my pipNames to select only the index that pertains to my Application Gateway based on name, like in the following:

var appGwName = 'appGw01'
module AppGw 'appgw.bicep' = {
  name: appGwName
  params: {
    PublicIpName: pipNames.First(pip => pip.name.Contains('${DeploymentPrefix}-ApplicationGateway-${resourceMapping['publicIp']}').name
  }
}

This would filter the pipNames to only those containing the string (based on how it was built) that matches that created for the Application Gateway. I could easily simplify this can just use the same named variable throughout for a common value as well. If there's a match, I get the name property. If not, it throws and I see that the deployment failed.

Describe the solution you'd really like
My proposal above would likely involve changes to ARM which isn't as likely to happen (though I've got to imagine they'd be well-received given how useful I'd find it). Rather than make those changes in ARM, again, all these values are known upfront (e.g. none is based on the output value of a module), so I'd like to see an extension of the constant folding idea applied to conditions such as this.

Specifically, identify LINQ operations as functions baked into Bicep itself and at build time, run this out in a what-if style fashion and determine what the values would be, then replace my function with those values so the actual ARM deployment is simply correct and using static values, and all the heavy lifting is just done on my own system at build-time.

For those values that are dependent on module outputs (which again cannot be worked out in advance), show an error at build indicating this isn't yet supported, but since this is the pipe dream section of the feature request, defer such projections to be evaluated by ARM at run-time where not possible to determine at build-time locally.

[alex-frankel]

Seems related to #4691 and #4153. Whit, can you take a look and see if either/both of those cover what you are asking for?

[WhitWaldo]

@alex-frankel Either of those issues would be excellent for this purpose. I'm curious though - to what extent could the ask be made simpler by introducing it in stages?

##Stage 1
If the inputs for each function are limited to those known at compile-time, could this simply be pre-computed as part of the build process and hide everything operationally from ARM via constant folding? Seems if you don't then have to make changes to ARM to push this through, it becomes a lot easier to move forward with the idea.

##Stage 2
As there's a clear path forward to using these operators in Stage 1 and work is being done to incorporate that in the build pipeline, also move forward with developing such functionality with the ARM team so this can be used with module outputs as well.

Couple it all with richer type inference and this could be a huge step forward for Bicep.

[alex-frankel]

Sounds good. I'm not sure how realistic the stage 1 shortcut is, but definitely worth considering.

Going to close this and track with the other issues.