Adding ES domain to VPC should force a new resource

[schmod]

If VPC configuration settings are added to an extant aws_elasticsearch_domain, Terraform should force the creation of a new domain, rather than attempting to modify the existing one in-place, as AWS Elasticsearch does not currently support migrating a public ES domain into a VPC.

For example, if I deploy the following configuration:

resource "aws_elasticsearch_domain" "main" {
  domain_name = "${var.name}"
  elasticsearch_version = "${var.version}"

  cluster_config {
    instance_type = "${var.node_type}"
    instance_count = "${var.instance_count}"
    dedicated_master_enabled = "${var.dedicated_master_enabled}"
    dedicated_master_count = "${var.dedicated_master_count}"
    dedicated_master_type = "${var.dedicated_master_type}"
    zone_awareness_enabled = "${var.zone_awareness_enabled}"
  }

  ebs_options {
    ebs_enabled = true
    volume_type = "gp2"
    volume_size = "${var.volume_size}"
  }
}

And subsequently modify the configuration and try to redeploy it:

resource "aws_elasticsearch_domain" "main" {
  domain_name = "${var.name}"
  elasticsearch_version = "${var.version}"

  cluster_config {
    instance_type = "${var.node_type}"
    instance_count = "${var.instance_count}"
    dedicated_master_enabled = "${var.dedicated_master_enabled}"
    dedicated_master_count = "${var.dedicated_master_count}"
    dedicated_master_type = "${var.dedicated_master_type}"
    zone_awareness_enabled = "${var.zone_awareness_enabled}"
  }

  ebs_options {
    ebs_enabled = true
    volume_type = "gp2"
    volume_size = "${var.volume_size}"
  }

  vpc_options {
    security_group_ids = ["${split(",", var.security_group_ids)}"]
    subnet_ids = ["${split(",", var.subnet_ids)}"]
  }
}

Error: Error applying plan:

1 error(s) occurred:

* module.elasticsearch.aws_elasticsearch_domain.main: 1 error(s) occurred:

* aws_elasticsearch_domain.main: ValidationException: Domain should be created with VPC options to update VPC options later
	status code: 400, request id: XXX

---

VPC support is new to AWS Elasticsearch, and has only been supported since v1.2.0 of terraform-provider-aws (see #2010 and #2013 for details)

[Ninir]

Hi @schmod

This has been taken care in #2157 and will be part of the next release.

Sorry for the noise here, Happy Terraforming!

[handlerbot]

@Ninir, I have some important feedback about #2157 as the original author of the VPC support for Elasticsearch; I do not think that ForceNew has the appropriate semantics for the goal here.

If you want to add a vpc_options block to an existing Elasticsearch domain (as in @schmod's use case) or remove a vpc_options block from an existing Elasticsearch domain, then yes, the entire Elasticsearch domain must be destroyed and recreated, and ForceNew is appropriate.

However, if you want to modify the fields of a vpc_options block in an existing Elasticsearch domain in VPC, that operation can be performed as an update on the existing resource, without requiring a full destroy/create cycle. Given that Elasticsearch is a data store (and that AWS doesn't have first-class API support for taking snapshots and restoring them, you have to do via-HTTP manual work), preserving the ability to update security groups and subnets via Terraform without forcing a rebuild seems important.

Is there a option similar to ForceNew that has these characteristics, or a reasonable and idiomatic way to implement that in Terraform?

[Ninir]

Hi @handlerbot

As you said, and due to the way ForceNew is implemented, this case will only ForceNew if:

• vpc_options is added to the configuration
• vpc_options is removed from the configuration

If one of the vpc_options field is updated, then the update is performed, but not the recreation!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!