Support nested resources with lexical scoping

[rynowak]

Is your feature request related to a problem? Please describe.

Specifying parent/child relationships can be confusing and verbose. You can also end up in situations where the child doesn't reference the parent and so have nothing to add dependsOn for you.

Describe the solution you'd like

The ability to nest resource definitions such that parent/child relationships are generated for you.

Child resources would be allowed to appear at the top level of nesting inside another resource. I'm showing the child resource as indented here, but there's no requirement that it be aligned, only that it doesn't appear inside a nested object definition.

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
        }

        resource child 'GrandchildType' = {
            name: 'child'
            properties: { 
            }
        }
    }
}

The examples here assume the answers to some of the questions that are still being discussed

Overview

The proposal allows the following simplifications in how these cases are written:

• The type name can be shortened since it must be based on the containing (parent) type
• The name property can be simplified because it must contain the parent's name
• The child resource has an automatic dependsOn the parent for creation-order purposes
• This simplifies loops where a resource contains children - no need to write two loops and join based on index
• if conditions can be omitted for child resources in this form

This proposal augments other proposals (like adding a parent property). The nested form of resource declarations can be seen as an opinionated combination of other features, like .parent.

Open Questions

• Close on syntax for referencing nested resources
• Whether to permit multi-generational families (direct nesting of grandchild or more)
• Do we allow qualified type names for children or just relative type names?

Parsing

With this proposal, resources may directly contain other resources. This must be at the top level - not nested inside an object declaration.

OK

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }
}

Error

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: {
      // error: not at top level of parent resource
      resource child 'ChildType' = {
          name: 'child'
          properties: { 
      }
  }
}

---

Lexically-nested resources my appear interspersed with other properties:

OK

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }   
}

// OR
resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    
    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }

   properties: { 
    }
}

// OR
resource parent 'Some.Resource/Type@01-01-2020' = {

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }

   name: 'parent'
   properties: { 
    }
}

Rationale: It doesn't really make sense to block this. We might think that the middle case from above is ugly, but if we allow the first and last case it makes sense to allow the middle one.

Scoping

Lexically nested resources declare names that are within the scope of their parent resource. There's a discussion happening right now about to refer to such resources. For the purpose of examples, I'm filling in the a/b/c syntax.

OK

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }   
}

// Assuming `a/b/c` syntax for children
output parent/child.name

Error

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }   
}

// error: child is not bound as a symbol at file-scope
output child.name

Type System

I suspect some of this will require debate and discussion, see open questions.

Within a resource, the nested types must be child types of the containing (parent) resource. Type names use a simplified format, and must refer to a single type segment. The API Version is optional, and will be assumed to match the parent if unspecified.

OK

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }   
}

Error

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    // Error: using qualified name
    resource child 'Some.Resource/ChildType@01-01-2020' = {
        name: 'child'
        properties: { 
    }   
}

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    // Error: multiple levels of ancestry
    resource grandchild 'ChildType/GrandChildType' = {
        name: 'grandchild'
        properties: { 
    }   
}

Rationale:

If we allow simplified (relative) type names and qualified type names then we have to be able to tell the difference between them. This is entangled with the decision to allow/ban mutli-generational nesting (directly containing a grandchild resource).

If we allow directly nesting a grandchild resource then the question of how do know the parent? becomes necessary to answer. We're automatically generating the full name of the resource, and injecting an dependsOn entry, so we need to know how to refer to the parent. We could choose to relax this restriction in the future by requiring the .parent property to be set, but that seems unnecessary since you can fall back to the more complex form, or use an existing reference to the parent as better options - remember this is about directly nesting a multi-generation relationship.

Now let's return to how type names are written:

• If we only allow relative type names ChildType vs Some.Resource/Type/ChildType then the task is simple
• If we allow both relative and qualified type names we have to be able to tell the difference
• A qualified type name syntactically looks like a multi-generational relative type name - Some.Resource/Type vs ChildType/GrandChildType - so allowing either of these constrains our ability to relax limitations in the future

---

For a nested resource, the .parent property is considered to be readonly.

Error

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
    }

    // Error: parent is readonly
    resource child 'Some.Resource/ChildType@01-01-2020' = {
        name: 'child'
        parent: ... 
        properties: { 
    }   
}

The lexical nesting has the same semantic effect as setting the parent, so we should not allow it to be manually set and cause a conflict by setting a different value.

Semantics and Codegen

The child resources are assumed to have an implement dependsOn to the parent. The cycle checker in the compiler will be made aware of this as well.

Error

resource parent 'Some.Resource/Type@01-01-2020' = {
    name: 'parent'
    properties: { 
        style: child.properties.style // Error: cycle detected
    }

    resource child 'Some.Resource/ChildType@01-01-2020' = {
        name: 'child'
        properties: {
          style: 'tank top'
          description: 'show off those ARMs'
        }
    }   
}

---

Loops and conditions on the parent resource carry:

// creates N instances of 'parent' and N instances of 'child'
resource parent 'Some.Resource/Type@01-01-2020' [for widget in widgets = {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }   
}]

// creates 0 instances of 'parent' and 0 instances of 'child'
// OR
// creates 1 instance of 'parent' and 1 instance of 'child'
resource parent 'Some.Resource/Type@01-01-2020' = if (create) {
    name: 'parent'
    properties: { 
    }

    resource child 'ChildType' = {
        name: 'child'
        properties: { 
    }   
}]

---

Existing has little overlap with the semantics here since nesting of resources just creates a reference. It 'just works'.

---

[rynowak]

/cc @alex-frankel @majastrz

[anthony-c-martin]

Team Discussion - 02/10/21 & 02/11/21

Comments/decisions

• We will limit the scope to just parent -> direct child relationships. We don't want to support multigenerational relationships (parent -> grandchild), but they should be expressible via parent -> child -> grandchild.
• Although we do not plan to tackle extension resources with this spec, we should mock it up to see if we can leave the door open to potentially implement this in the future.
• We don't want to allow fully-qualified types in the nested child declarations - instead we should only allow a single type string with optional API version (either 'type' or 'type@2020-01-01')
• In the initial implementation, we will restrict to just a single resource loop - looping over 2+ resources in the hierarchy will not be supported.
• We do not see any compatibility issues with the for, if or existing features.
• We agree that automatic dependsOn insertion makes sense.

Detailed discussion / open questions

How should we reference inner-scoped child resources in the outer scope?

We discussed a number of options for referring to a child resource in an outer scope. Given the below:

resource myParent 'My.Rp/parent@2020-01-01' = {
  name: 'parent'
  resource myChild 'child' = {
    name: 'child'
  }
}

Some options we considered:

1. Use the . operator, and access the child as if it were any other property on the parent: myParent.myChild
   • • Intuitive and easily discoverable via intellisense.
   • • Potentially confusing semantics when selecting resource names which shadow existing properties - e.g. naming a child name or properties.
2. Have a dedicated property to access children: myParent.resources.myChild
   • • Intuitive and easily discoverable via intellisense.
   • • Question of whether we can safely reserve .resources and ensure it can't be used for a legitimate resource property. An alternative of .$resources was also suggested.
   • • Will lead to quite verbose referencing - something we are keen to avoid. e.g. myParent.resources.myChild.properties.someProp
3. Use an entirely different symbol to access children.
   • Some options we discussed:
     • myParent/myChild - familiar to anyone used to ARM's resourceId format. Possible concerns over lexing/parsing as / is already used for division.
     • myParent:myChild
     • myParent>myChild
     • myParent..myChild
   • There were concerns over discoverability for users only familiar with . property access. One possibility is to offer myParent:myChild with intellisense at an outer scope rather than waiting for someone to type myParent: before suggesting it.

It's important to consider how all of these proposals would work with loops - e.g. myParent[0]:myChild - although we believe all should be compatible, there are implications for intellisense.

We would like community feedback on this, as it's largely a question of personal preference.

Some alternatives we dismissed:

1. Allow referencing myChild directly. We decided this would not play well with loops, and introduces confusing scoping rules, so did not consider this.

What is compatibility like with the current child resource declaration syntax, and the proposed parent property syntax (#127)?

// 1. example of the current
resource myParent 'My.Rp/parent@2020-01-01' = {
  name: 'myParent'
  ...
}
resource myChild 'My.Rp/parent/child@2020-01-01' = {
  name: '${myParent}/myChild'
  ...
}

// 2. example of the 'nested' syntax
resource myParent 'My.Rp/parent@2020-01-01' = {
  name: 'myParent'
  ...
  resource myChild 'child' = {
    name: 'myChild'
    ...
  }
}

// 3. example of the 'parent' syntax
resource myParent 'My.Rp/parent@2020-01-01' = {
  name: 'myParent'
  ...
}
resource myChild 'My.Rp/parent/child@2020-01-01' = {
  parent: myParent
  name: 'myChild'
  ...
}

1. We had consensus on removing the current syntax for declaring a child resource:
   • Using string interpolation to join parent + child name is confusing, easy to get wrong, and leads to problems with type validation (Bicep types are not generated for resources that use "name" as a polymorphic discriminator #657)
   • While it's obvious in syntax 2 & 3 that the value of myChild.name should give you 'myChild', you would expect syntax 1 to give 'myParent/myChild'. This introduces a disconnect between the two syntaxes which is difficult to reason about:
     • There's scope for subtle bugs to be introduced when refactoring between syntaxes 1 & 2.
     • Most of the time, it's much more useful to be able to just get the specific resource name rather than the fully-qualified resource name.
   • There is an obvious mechanism to refactor between 2 <--> 3 - with 2 you can imagine the nested syntax as implicitly setting the parent property. We should also be able to offer a code action to automate it.
2. Before removing the current syntax, we need to think of a more efficient way to declare an existing resource, so that a single-declaration child resource is still possible in cases where the parent is not being deployed:

   resource myChild 'My.Rp/parent/child@2020-01-01' = {
     parent: resource('My.Rp/parent', 'parent')
     name: 'myChild'
   }

tl;dr

• This proposal looks good.
• Action items:
  • Close on the exact syntax for referencing a nested child resource - preferably with some community feedback.
  • Mockup how nesting extension resources would fit with this proposal.
  • Come up with a one-liner syntax proposal for adding a reference to an existing parent resource, and update Improve declarations of child resources #127 with it.