"Must Contain Tags" Not working

[anthonycolon25]

I have some resources in a main.tf file which support tags, however even if tags are not supplied the following scenario still passes:

Feature: Resources should be properly tagged
  In order to keep track of resource ownership
  As engineers
  We'll enforce tagging on all resources

  Scenario: Ensure all resources have tags
    Given I have resource that supports tags defined
    Then it must contain tags

These is what's in the main.tf file:

provider "aws" {
  #   access_key = “aws_access_key_id”
  #   secret_key = “aws_secret_access_key_id”
  region = "us-east-1"
}

data "aws_availability_zones" "all" {
}

### Creating EC2 instance
resource "aws_instance" "web" {
  ami                    = var.amis[var.region]
  count                  = var.count_var
  key_name               = var.key_name
  vpc_security_group_ids = [aws_security_group.instance.id]
  source_dest_check      = false
  instance_type          = "t2.micro"
#   tags = {
#     Name = format("web-%03d", count.index + 1)
#   }
}

### Creating Security Group for EC2
resource "aws_security_group" "instance" {
  name = "terraform-example-instance"
  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
    # cidr_blocks = ["10.0.0.0/8"]
  }
}

## Creating Launch Configuration
resource "aws_launch_configuration" "example" {
  image_id        = var.amis[var.region]
  instance_type   = "t2.micro"
  security_groups = [aws_security_group.instance.id]
  key_name        = var.key_name
  user_data       = <<-EOF
              #!/bin/bash
              echo "Hello, World" > index.html
              nohup busybox httpd -f -p 8080 &
EOF


  lifecycle {
    create_before_destroy = true
  }
}

## Creating AutoScaling Group
resource "aws_autoscaling_group" "example" {
  launch_configuration = aws_launch_configuration.example.id
  availability_zones = data.aws_availability_zones.all.names
  min_size = 2
  max_size = 10
  load_balancers = [aws_elb.example.name]
  health_check_type = "ELB"
#   tag {
#     key = "Name"
#     value = "terraform-asg-example"
#     propagate_at_launch = true
#   }
}

## Security Group for ELB
resource "aws_security_group" "elb" {
  name = "terraform-example-elb"
  egress {
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

#   tags = {
#     Name = "allow_all"
#   }
}

### Creating ELB
resource "aws_elb" "example" {
  name = "terraform-asg-example"
  security_groups = [aws_security_group.elb.id]
  availability_zones = data.aws_availability_zones.all.names
  health_check {
    healthy_threshold = 2
    unhealthy_threshold = 2
    timeout = 3
    interval = 30
    target = "HTTP:8080/"
  }
  listener {
    lb_port = 80
    lb_protocol = "http"
    instance_port = "8080"
    instance_protocol = "http"
  }
}

Here is the output from the run:

$ terraform-compliance -p /target/tc-test2/tf-template/plan.out.json -f /target/tc-test2/features
terraform-compliance v1.0.7 initiated

* Features  : /target/tc-test2/features
* Plan File : /target/tc-test2/tf-template/plan.out.json

. Running tests.
Feature: Resources should be properly tagged  # /target/tc-test2/features/tags.feature
    In order to keep track of resource ownership
    As engineers
    We'll enforce tagging on all resources

    Scenario: Ensure all resources have tags
        Given I have resource that supports tags defined
        Then it must contain tags

Am I missing anything here?

Thanks in advance.

[anthonycolon25]

I looked at the json output and it seems that even if the tags are not defined in the tf file there is still a tags object with the value of null

,"tag":[],"tags":null,

So this behavior may be correct.

[anthonycolon25]

I think I worked around it by doing the following which checks if the value is not null

  Scenario: Ensure all resources have tags
    Given I have resource that supports tags defined
    When it contains tags
    Then its value must not match the "\x00" regex

[eerkunt]

Very nice find.

I think we may need a step that will trigger your last step, something like ;

Then its value <must/must not> be null

[eerkunt]

This is implemented on 1.0.11. Could you please give a try with this version, if possible ? Thanks!

[anthonycolon25]

I have tested 1.0.11. There still seems to be something going. I'll try to explain.

With 1.0.11 the following two scenarios behave the same:

   Scenario: Ensure all resources have tags
    Given I have resource that supports tags defined
    When it contains tags
    Then its value must not match the "\x00" regex

and

   Scenario: Ensure all resources have tags
    Given I have resource that supports tags defined
    When it contains tags
    Then its value must not be null

I now have many more scenarios than when I opened this issue. At the moment I have tags on all taggable resources used in my tf file. When I run the test, the first scenario below still fails but the second one passes.

Feature: Resources should be properly tagged  # /target/tc-test2/features/tags.feature
    In order to keep track of resource ownership
    As engineers
    We'll enforce tagging on all resources

    Scenario: Ensure all resources have tags
        Given I have resource that supports tags defined
        When it contains tags
        Then its value must not be null
          Failure: tags property in aws_security_group.elb resource matches with Null/None regex. It is set to None.

    Scenario: Ensure that AppEnvironment tag is defined
        Given I have resource that supports tags defined
        When it contains tags
        Then it must contain AppEnvironment
        And its value must match the "^(sandbox|dev|deva|devb|ita|itb|qa|pv|prod|poc|training)$" regex

I am not sure how it thinks that tags are set to None when the security group definitely has tags set. I am attaching plan.out json fro review.
plan.out.json.txt

[eerkunt]

This looks like a bug to me, trying to fix it.

[eerkunt]

Could you please try again with 1.0.14 ?

[eerkunt]

Assuming this is fixed, please re-open the issue if this is not working for you.