New Resource: aws_custom_resource

[scottwinkler]

Introduction

Lambda-backed Custom resources are indispensable for bridging the gap of infrastructure not currently supported by terraform, without having to go through the effort of writing a custom plugin or extending existing providers. Previously, custom resources are supported through CloudFormation but not terraform, so the old workaround was to call a cloudformation script from terraform thats whole purpose is to call the custom resource, but that is an anti-pattern that needlessly makes custom resources more difficult to use.

How to Use

Inputs

• service_token - (Required) The arn of your lambda function to invoke
• resource_type - (Required) The name of your custom resource
• resource_properties - (Required) A map[string]string of parameters for the customresource

Outputs

• data - A map[string] string of outputs

This is an example of how the quick certificate could be used:

resource "aws_custom_resource" "test" {
 
  service_token = "${aws_lambda_function.my_lambda.arn}"
 
  resource_type = "CustomTest"
 
  resource_properties = {
 
    a = "1"
 
    b = "1"
 
  }
 
}
}

When you run "terraform apply", your lambda function will be invoked with a json event as follows. RequestType will be one of Create||Update||Delete, and it is up to you to decide how to handle that. In this case, it is "Create". In addition, you will receive the ResourceType, which is defined by you, and can be used to have a single lambda function responsible for handling the creation of multiple custom resources. Finally, you will receive the ResourceProperties which is a map of input parameters.

{
    "RequestType": "Create",
    "ResourceType": "CustomTest",
    "OldResourceProperties": {
        "a": "1",
        "b": "1"
    },
    "ResourceProperties": {
        "a": "1",
        "b": "1"
    }
}

Similarly, if you change the resource properties and run terraform apply, your lambda function will get called with an "Update" event that contains information about the OldResourceProperties, as well as the new ResourceProperties. In this example, b was changed from 1 to 2, and this is the event that was generated

{
    "RequestType": "Update",
    "ResourceType": "CustomTest",
    "OldResourceProperties": {
        "a": "1",
        "b": "1"
    },
    "ResourceProperties": {
        "a": "1",
        "b": "2"
    }
}

Delete is the final event that is created, and is run on "terraform destroy". It is up to you to figure out how to clean up the resources you created. Below is an example of a "Delete" event.

{
    "RequestType": "Delete",
    "ResourceType": "CustomTest",
    "OldResourceProperties": {
        "a": "1",
        "b": "2"
    },
    "ResourceProperties": {
        "a": "1",
        "b": "2"
    }
}

In your lambda function, you are expected to return a response to the outputstream with a json struct like follows. Status is required and must be either "SUCCESS" or "FAILURE". If "FAILURE", then a Reason must be specified, which will be printed in the terraform console as an error message. Finally, Data is a map[string]string that will be saved and can be used as an output from the module. i.e ${aws_custom_resource.test.data[“out1”] }.

{  "Status" : "SUCCESS",  "Reason" : "",  "Data" : {       "out1" : "val1",       "out2" : "val2"  }}

Below is a trivial example of the source code for a lambda function that handles the customresource.

public class Hello implements RequestStreamHandler {
    public void handleRequest(InputStream request, OutputStream outputStream, Context context) throws IOException {
        System.out.println("received request");
        Gson gson = new Gson();
        JsonObject inputObj;
        JsonObject data = new JsonObject();
        try {
            JsonParser parser = new JsonParser();
            inputObj = parser.parse(IOUtils.toString(request, StandardCharsets.UTF_8)).getAsJsonObject();
            System.out.println("input: " + gson.toJson(inputObj));
            System.out.println("performing operation: " + inputObj.get("RequestType").getAsString()); //prints CREATE||UPDATE||DELETE
            data.addProperty("out1",inputObj.get("ResourceProperties").getAsJsonObject().get("a").getAsString()); //read from the input variable and sent it to the output
 
        } catch (IOException e) {
            e.printStackTrace();
        }
        System.out.println("writing to output stream");
        JsonObject jsonObject = new JsonObject();
        jsonObject.addProperty("Status","SUCCESS"); // add success condition
        jsonObject.addProperty("Reason","You suck"); //add reason for failure (optional)
        jsonObject.add("Data",data); //add map[string] string to data
 
        System.out.println(gson.toJson("output: " + jsonObject)); //sanity check
        outputStream.write( gson.toJson(jsonObject).getBytes()); //write to outputstream
    }
}

[Bhuwan]

Nice one @scottwinkler looking for this in my own usage.
Anything I can assist with to get this into baseline?

[scottwinkler]

Thanks! @Bhuwan I could use some help writing test cases... even simple, incomplete ones would be good. Also the error handling is not perfect, and needs improvement, but it does work well enough and I have been using this resource with great success at my company. If you want to use this right away, I can provide you a stripped down custom provider which only has this one resource in it, and an example

[Bhuwan]

@scottwinkler Sorry, haven't had the chance to assist. But, in the meantime can you provide a the stripped down version with an example?

[Bhuwan]

I think this is now completed and can be closed based on https://www.terraform.io/docs/providers/aws/d/lambda_invocation.html and #4222

[scottwinkler]

No it's not. That is a data resource, not a full fledged terraform resource. It might be useful to some people but it does not replace the need for this

[nathanagez]

Hi, any status on this PR @scottwinkler @aeschright @radeksimko ?

[scottwinkler]

I would be happy to see this merged but I dont know if it will be anytime soon.

In the meantime, I would recommend taking a look at the shell provider I wrote. It serves the same purpose. https://github.com/scottwinkler/terraform-provider-shell

[nathanagez]

I will take a look with the team, thank you :)
Do you know why it still not merged ? It's been since 2018...

[scottwinkler]

🤷‍♂️ no idea

[alanmpitts]

Any update on this PR? It would be useful for me.

[numToStr]

Any update?

[ewbankkit]

Replaced by #10096.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!