Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_eventgrid_topic - support for input_schema, input_mapping_fields, and input_mapping_default_values #6858

Merged
merged 4 commits into from
May 26, 2020
Merged

azurerm_eventgrid_topic - support for input_schema, input_mapping_fields, and input_mapping_default_values #6858

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5eb1dfe
r/evengrid_topic_resource: input schema support
jrauschenbusch May 9, 2020
04f79d8
fix: terrafmt
jrauschenbusch May 12, 2020
5681272
fix: add map length check
jrauschenbusch May 14, 2020
617f1f8
fix: lintrest
jrauschenbusch May 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 237 additions & 1 deletion azurerm/internal/services/eventgrid/eventgrid_topic_resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,85 @@ func resourceArmEventGridTopic() *schema.Resource {

"resource_group_name": azure.SchemaResourceGroupName(),

"input_schema": {
Type: schema.TypeString,
Optional: true,
Default: string(eventgrid.InputSchemaEventGridSchema),
ForceNew: true,
ValidateFunc: validation.StringInSlice([]string{
string(eventgrid.InputSchemaCloudEventSchemaV10),
string(eventgrid.InputSchemaCustomEventSchema),
string(eventgrid.InputSchemaEventGridSchema),
}, false),
},

"input_mapping_fields": {
Type: schema.TypeList,
Optional: true,
MaxItems: 1,
ForceNew: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"id": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we try and force a user to pass in at least one of these attributes? I'm wondering what happens if a user doesn't pass anything into this for whatever reason. We could add

AtLeastOneOf: []string{"input_mapping_files.0.id", "input_mapping_files.0.topic".....}

to achieve this

Copy link
Contributor Author

@jrauschenbusch jrauschenbusch May 14, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with that. It would be really nice to give the end user a hint that there might be a misuse, but AtLeastOneOf conflicts with the Optional flag. However, it must be optional because the input_mapping is only used when the input_schema is changed to CustomEventSchema. EventGridSchema is used by default. In this case, an input_mapping is just ignored.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made some tests now.

When using EventGridSchema or CloudEventV01Schema as input_schema, and we add an (empty) input_mapping_fields and/or input_mapping_default_values block, the following error will be reported:

Error: eventgrid.TopicsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidRequest" Message="InputSchemaMapping must not be provided when InputSchema is EventGridSchema or CloudEventV01Schema."

When using CustomEventSchema as input_schema and adding empty input_mapping_fields and input_mapping_default_values blocks the following will be reported:

Error: eventgrid.TopicsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidRequest" Message="Invalid InputSchemaMapping: DataVersion entry cannot be null, and either sourceField or defaultValue must be specified."

After adding a sourceField or defaultValue for data_version the following is reported:

Error: eventgrid.TopicsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidRequest" Message="Invalid InputSchemaMapping: Invalid Subject. Either sourceField or defaultValue should be specified."

After adding a sourceField or defaultValue for subject the following is reported:

Error: eventgrid.TopicsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidRequest" Message="Invalid InputSchemaMapping: Invalid EventType. Either sourceField or defaultValue should be specified."

After adding a sourceField or defaultValue for event_type the resource could finally be created.

Hence at Plan time we would need the following checks:

  • if input_schema != "CustomEventSchema", then the blocks input_mapping_fields and input_mapping_default_values must not be present
  • if input_schema == "CustomEventSchema", then subject, event_type and data_version are required in one of the input_mapping(_default) blocks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I have an idea to make this easier on users and it's to split these attributes out into their own respective lists. InputSchemaCloudEventSchemaV10 and InputSchemaEventGridSchema can just be booleans that conflict with each other like is_event_grid_schema and then have custom_event_schema_input_mapping with all of these variables in there. That will let us better differentiate the Required and Optional attributes. While being harder from a maintainer perspective, it should really help users see how all of the pieces here fit together.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is making things even more complex, also for the users. As there are not only input_mapping_fields to be considered but also input_mapping_default_values.

I think your design proposal would be look like:

resource "azurerm_evengrid_topic" "example" {

   ...

   enable_event_grid_schema = true // Optional, Default,  ConflictsWith ["enable_cloud_event_schema_v10", "custom_event_schema_input_mapping"]

   enable_cloud_event_schema_v10 = true  // Optional, ConflictsWith ["enable_event_grid_schema", "custom_event_schema_input_mapping"]

   custom_event_schema_input_mapping { // Optional,  ConflictsWith ["enable_event_grid_schema", "enable_cloud_event_schema_v10"]
      mapping_fields {
        id = "path.id" // optional
        topic= "path.topic" // optional
        event_time= "path.eventtime" // optional
        subject = "path.subject" // required either here
        event_type = "path.type" //  required either here
        data_version = "path.version"  //  required either here
      }
      default_mappings {
        subject = "foo" // or required here
        event_type = "bar" // or required here
        data_version = "" // or required here
      }
   }
}

So far I've aligned the design with the existing resource azurerm_eventgrid_domain. (see https://www.terraform.io/docs/providers/azurerm/r/eventgrid_domain.html#input_schema)

LMKWYT

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your design is what I was thinking but you're right that the burden is a little too high on both sides. I appreciate that you took the time to write it out though.

I wasn't aware that azurerm_eventgrid_domain also did something similar so I'm a fan of keeping things consistent between that resource and this one.

Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"topic": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"event_time": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"event_type": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"subject": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"data_version": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
},
},
},

"input_mapping_default_values": {
Type: schema.TypeList,
Optional: true,
MaxItems: 1,
ForceNew: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"event_type": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"subject": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
"data_version": {
Type: schema.TypeString,
ForceNew: true,
Optional: true,
},
},
},
},

"endpoint": {
Type: schema.TypeString,
Computed: true,
Expand Down Expand Up @@ -97,9 +176,14 @@ func resourceArmEventGridTopicCreateUpdate(d *schema.ResourceData, meta interfac
location := azure.NormalizeLocation(d.Get("location").(string))
t := d.Get("tags").(map[string]interface{})

topicProperties := &eventgrid.TopicProperties{
InputSchemaMapping: expandAzureRmEventgridTopicInputMapping(d),
InputSchema: eventgrid.InputSchema(d.Get("input_schema").(string)),
}

properties := eventgrid.Topic{
Location: &location,
TopicProperties: &eventgrid.TopicProperties{},
TopicProperties: topicProperties,
Tags: tags.Expand(t),
}

Expand Down Expand Up @@ -147,6 +231,27 @@ func resourceArmEventGridTopicRead(d *schema.ResourceData, meta interface{}) err

return fmt.Errorf("Error making Read request on EventGrid Topic '%s': %+v", id.Name, err)
}
if props := resp.TopicProperties; props != nil {
d.Set("endpoint", props.Endpoint)

d.Set("input_schema", string(props.InputSchema))

inputMappingFields, err := flattenAzureRmEventgridTopicInputMapping(props.InputSchemaMapping)
if err != nil {
return fmt.Errorf("Unable to flatten `input_schema_mapping_fields` for EventGrid Topic %q (Resource Group %q): %s", id.Name, id.ResourceGroup, err)
}
if err := d.Set("input_mapping_fields", inputMappingFields); err != nil {
return fmt.Errorf("Error setting `input_schema_mapping_fields` for EventGrid Topic %q (Resource Group %q): %s", id.Name, id.ResourceGroup, err)
}

inputMappingDefaultValues, err := flattenAzureRmEventgridTopicInputMappingDefaultValues(props.InputSchemaMapping)
if err != nil {
return fmt.Errorf("Unable to flatten `input_schema_mapping_default_values` for EventGrid Topic %q (Resource Group %q): %s", id.Name, id.ResourceGroup, err)
}
if err := d.Set("input_mapping_default_values", inputMappingDefaultValues); err != nil {
return fmt.Errorf("Error setting `input_schema_mapping_fields` for EventGrid Topic %q (Resource Group %q): %s", id.Name, id.ResourceGroup, err)
}
}

keys, err := client.ListSharedAccessKeys(ctx, id.ResourceGroup, id.Name)
if err != nil {
Expand Down Expand Up @@ -196,3 +301,134 @@ func resourceArmEventGridTopicDelete(d *schema.ResourceData, meta interface{}) e

return nil
}

func expandAzureRmEventgridTopicInputMapping(d *schema.ResourceData) *eventgrid.JSONInputSchemaMapping {
imf, imfok := d.GetOk("input_mapping_fields")

imdv, imdvok := d.GetOk("input_mapping_default_values")

if !imfok && !imdvok {
return nil
}

jismp := eventgrid.JSONInputSchemaMappingProperties{}

if imfok {
mappings := imf.([]interface{})
mapping := mappings[0].(map[string]interface{})

if id := mapping["id"].(string); id != "" {
jismp.ID = &eventgrid.JSONField{SourceField: &id}
}

if eventTime := mapping["event_time"].(string); eventTime != "" {
jismp.EventTime = &eventgrid.JSONField{SourceField: &eventTime}
}

if topic := mapping["topic"].(string); topic != "" {
jismp.Topic = &eventgrid.JSONField{SourceField: &topic}
}

if dataVersion := mapping["data_version"].(string); dataVersion != "" {
jismp.DataVersion = &eventgrid.JSONFieldWithDefault{SourceField: &dataVersion}
}

if subject := mapping["subject"].(string); subject != "" {
jismp.Subject = &eventgrid.JSONFieldWithDefault{SourceField: &subject}
}

if eventType := mapping["event_type"].(string); eventType != "" {
jismp.EventType = &eventgrid.JSONFieldWithDefault{SourceField: &eventType}
}
}

if imdvok {
mappings := imdv.([]interface{})
mapping := mappings[0].(map[string]interface{})

if dataVersion := mapping["data_version"].(string); dataVersion != "" {
jismp.DataVersion = &eventgrid.JSONFieldWithDefault{DefaultValue: &dataVersion}
}

if subject := mapping["subject"].(string); subject != "" {
jismp.Subject = &eventgrid.JSONFieldWithDefault{DefaultValue: &subject}
}

if eventType := mapping["event_type"].(string); eventType != "" {
jismp.EventType = &eventgrid.JSONFieldWithDefault{DefaultValue: &eventType}
}
}

jsonMapping := eventgrid.JSONInputSchemaMapping{
JSONInputSchemaMappingProperties: &jismp,
InputSchemaMappingType: eventgrid.InputSchemaMappingTypeJSON,
}

return &jsonMapping
}

func flattenAzureRmEventgridTopicInputMapping(input eventgrid.BasicInputSchemaMapping) ([]interface{}, error) {
if input == nil {
return nil, nil
}
result := make(map[string]interface{})

jsonValues, ok := input.(eventgrid.JSONInputSchemaMapping)
if !ok {
return nil, fmt.Errorf("Unable to read JSONInputSchemaMapping")
}
props := jsonValues.JSONInputSchemaMappingProperties

if props.EventTime != nil && props.EventTime.SourceField != nil {
result["event_time"] = *props.EventTime.SourceField
}

if props.ID != nil && props.ID.SourceField != nil {
result["id"] = *props.ID.SourceField
}

if props.Topic != nil && props.Topic.SourceField != nil {
result["topic"] = *props.Topic.SourceField
}

if props.DataVersion != nil && props.DataVersion.SourceField != nil {
result["data_version"] = *props.DataVersion.SourceField
}

if props.EventType != nil && props.EventType.SourceField != nil {
result["event_type"] = *props.EventType.SourceField
}

if props.Subject != nil && props.Subject.SourceField != nil {
result["subject"] = *props.Subject.SourceField
}

return []interface{}{result}, nil
}

func flattenAzureRmEventgridTopicInputMappingDefaultValues(input eventgrid.BasicInputSchemaMapping) ([]interface{}, error) {
if input == nil {
return nil, nil
}
result := make(map[string]interface{})

jsonValues, ok := input.(eventgrid.JSONInputSchemaMapping)
if !ok {
return nil, fmt.Errorf("Unable to read JSONInputSchemaMapping")
}
props := jsonValues.JSONInputSchemaMappingProperties

if props.DataVersion != nil && props.DataVersion.DefaultValue != nil {
result["data_version"] = *props.DataVersion.DefaultValue
}

if props.EventType != nil && props.EventType.DefaultValue != nil {
result["event_type"] = *props.EventType.DefaultValue
}

if props.Subject != nil && props.Subject.DefaultValue != nil {
result["subject"] = *props.Subject.DefaultValue
}

return []interface{}{result}, nil
}
49 changes: 49 additions & 0 deletions azurerm/internal/services/eventgrid/tests/eventgrid_topic_resource_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,29 @@ func TestAccAzureRMEventGridTopic_requiresImport(t *testing.T) {
})
}

func TestAccAzureRMEventGridTopic_mapping(t *testing.T) {
data := acceptance.BuildTestData(t, "azurerm_eventgrid_topic", "test")

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acceptance.PreCheck(t) },
Providers: acceptance.SupportedProviders,
CheckDestroy: testCheckAzureRMEventGridTopicDestroy,
Steps: []resource.TestStep{
{
Config: testAccAzureRMEventGridTopic_mapping(data),
Check: resource.ComposeTestCheckFunc(
testCheckAzureRMEventGridTopicExists(data.ResourceName),
resource.TestCheckResourceAttr(data.ResourceName, "input_mapping_fields.0.topic", "test"),
resource.TestCheckResourceAttr(data.ResourceName, "input_mapping_fields.0.topic", "test"),
resource.TestCheckResourceAttr(data.ResourceName, "input_mapping_default_values.0.data_version", "1.0"),
resource.TestCheckResourceAttr(data.ResourceName, "input_mapping_default_values.0.subject", "DefaultSubject"),
),
},
data.ImportStep(),
},
})
}

func TestAccAzureRMEventGridTopic_basicWithTags(t *testing.T) {
data := acceptance.BuildTestData(t, "azurerm_eventgrid_topic", "test")

Expand Down Expand Up @@ -174,6 +197,32 @@ resource "azurerm_eventgrid_topic" "import" {
`, template)
}

func testAccAzureRMEventGridTopic_mapping(data acceptance.TestData) string {
return fmt.Sprintf(`
provider "azurerm" {
features {}
}
resource "azurerm_resource_group" "test" {
name = "acctestRG-%d"
location = "%s"
}
resource "azurerm_eventgrid_topic" "test" {
name = "acctesteg-%d"
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
input_schema = "CustomEventSchema"
input_mapping_fields {
topic = "test"
event_type = "test"
}
input_mapping_default_values {
data_version = "1.0"
subject = "DefaultSubject"
}
}
`, data.RandomInteger, data.Locations.Primary, data.RandomInteger)
}

func testAccAzureRMEventGridTopic_basicWithTags(data acceptance.TestData) string {
// currently only supported in "West Central US" & "West US 2"
location := "westus2"
Expand Down
31 changes: 31 additions & 0 deletions website/docs/r/eventgrid_topic.html.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,38 @@ The following arguments are supported:

* `location` - (Required) Specifies the supported Azure location where the resource exists. Changing this forces a new resource to be created.

* `input_schema` - (Optional) Specifies the schema in which incoming events will be published to this domain. Allowed values are `CloudEventSchemaV1_0`, `CustomEventSchema`, or `EventGridSchema`. Defaults to `EventGridSchema`. Changing this forces a new resource to be created.

* `input_mapping_fields` - (Optional) A `input_mapping_fields` block as defined below.

* `input_mapping_default_values` - (Optional) A `input_mapping_default_values` block as defined below.

* `tags` - (Optional) A mapping of tags to assign to the resource.
---

A `input_mapping_fields` supports the following:

* `id` - (Optional) Specifies the id of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `topic` - (Optional) Specifies the topic of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `event_type` - (Optional) Specifies the event type of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `event_time` - (Optional) Specifies the event time of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `data_version` - (Optional) Specifies the data version of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `subject` - (Optional) Specifies the subject of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

---

A `input_mapping_default_values` supports the following:

* `event_type` - (Optional) Specifies the default event type of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `data_version` - (Optional) Specifies the default data version of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

* `subject` - (Optional) Specifies the default subject of the EventGrid Event to associate with the domain. Changing this forces a new resource to be created.

## Attributes Reference

Expand Down