RDS - storage_encrypted = true does not work

[hashibot]

This issue was originally opened by @victorkabdebon as hashicorp/terraform#16894. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Hi Terraform team,

Please see below an issue that I found on the latest version of Terraform over the weekend.

Terraform Version

$ terraform --version
Terraform v0.11.1
+ provider.aws v1.1.0

Expected Behavior

I would have expected the database to be encrypted & setup. Instead, encryption was false.

Actual Behavior

In the output generated the flag was clearly set to storage_encrypted = true, but after it was created both in the admin and the tfstate the encryption was set to false.

Even worse, terraform would get stuck in a loop where at each "apply" it would want to recreate my database every time (probably because the program saw the encryption set to false in the state).

Steps to Reproduce

Truncated terraform configuration

resource "aws_db_instance" "default" {
  engine = "postgres"
  engine_version = "9.6.3"

  multi_az = false 
  backup_retention_period = 10
  auto_minor_version_upgrade = true

  storage_encrypted = true
}

As far as I can tell, turning storage_encrypted to true is enough, then running terraform apply.

[victorkabdebon]

By the way, it seems that setting up automatic backup is also broken.

[atsushi-ishibashi]

docs says

StorageEncrypted

Specifies whether the DB instance is encrypted.

Amazon Aurora

Not applicable. The encryption for DB instances is managed by the DB cluster. For more information, see CreateDBCluster.

[aleksandersumowski]

@atsushi-ishibashi: setting the storage_encrypted at aurora cluster level doesn't seem to work for me as well. I'll create a separate issue for that ...

[victorkabdebon]

It basically boils down to the fact that if an option is not "consistent" (as in it would not be possible to create an instance with these parameters from the interface), Amazon will create something without the right parameters (my guess is the parameters are ignored). It feels like an incorrect design choice for something that sensitive as it gives you a false feeling of success.

Anyway, the way to "fix" that is to go through the interface and see what options will work together and tweak your terraform configuration

[emulanob]

I'm having the same issue, but in my case the RDS instance is being created from an existing non-encrypted snapshot. Also, I'm informing the kms_key_id with the ARN for the master key I want to use on the encryption.

The result is a working RDS Instance without encryption, and any new terraform plan will inform the need of a new instance as the existing one is not encrypted.

[zdevi]

Up, problem is still here. storage_encrypted = true doesn't do anything.

[emulanob]

Note: my case is solved. It takes an encrypted snapshot to restore a encrypted RDS instance. If you try to restore an unencrypted snapshot to a new instance, does not matter if you enable encryption or not, it will be unencrypted as well.

But for brand new instances (@victorkabdebon 's case) it should enable encryption just fine, which is not happening.

[artkrz]

Wow, reported back in 2017 and still not resolved :(

[mhumeSF]

Agreed. I feel there could be a check of the snapshot is encrypted and error if not when storage_encrypted = true and restoring from snapshot.

[mhumeSF]

Dropping here for ref; but I think a check could be added to snapshot_identifier in these files

aws/resource_aws_docdb_cluster.go
aws/resource_aws_neptune_cluster.go
aws/resource_aws_rds_cluster.go
aws/resource_aws_db_instance.go
aws/resource_aws_redshift_cluster.go

[mhumeSF]

Would also be cool if kms_key_id were specified to verify snapshot restore is encrypted with same kms_key_id and error if not

[rios0rios0]

+1 up

[HannesBBR]

The same issue seems to be present when restoring a DocumentDB cluster from an unencrypted snapshot. Even though storage_encrypted=true is specified, the end-result after terraform apply is a cluster without encryption-at-rest enabled. This then causes subsequent plans to want to recreate the cluster.

[bill-rich]

It sounds like the cases where storage_encrypted is not working are because of conflicting options in the configuration. When the resource cannot be created as specified, the ideal situation would be that the API returns an error. Returning an error would prevent any confusion like what we see in cases like this.

Although it would be possible to add checks into the provider to prevent some cases, we generally try to leave that to the API. One reason is that if the functionality changes on the API side, we do not want to provider to blocking supported features. More importantly, it is not really practical to maintain the logic that would be required validate all working configurations for a resource.

I'd recommend testing your aws_db_instance configuration before adding it to a larger deployment, or before deploying it to a critical environment. That would allow for running apply more than once, and being able to adjust the configuration if the resource needs to be recreated.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!