packer Post Processor Ignores User Variables in "only"

[ankitkl]

{
"variables": {
    "aws_region": "ap-southeast-1",
    "post": "amazon-ebs"
  },

  "description": "Amazon AMI's",
  "builders": [{
    "type": "amazon-ebs",
    "region":  "{{user `aws_region`}}" ,
    "vpc_id": "vpc-xxxxx",
    "subnet_id": "subnet-4xxxxxxx",
    "security_group_id": "xxxxxxxxxx",
    "source_ami": "ami-2cxxxxx",
    "instance_type": "t2.micro",
    "ssh_username": "ec2-user",
    "ami_name": "packer-example-{{timestamp}}",
    "tags": {
        "Amazon_AMI_Management_Identifier": "packer-example"
    }
  },
  {
  "type": "virtualbox-iso",
  "guest_os_type": "Ubuntu_64",
  "iso_url": "http://releases.ubuntu.com/12.04/ubuntu-12.04.5-server-amd64.iso",
  "iso_checksum": "769474248a3897f4865817446f9a4a53",
  "iso_checksum_type": "md5",
  "ssh_username": "packer",
  "ssh_password": "packer",
  "shutdown_command": "echo 'packer' | sudo -S shutdown -P now"
}
],
  "provisioners":[{
    "type": "shell",
    "inline": [
      "echo hostname",
      "sudo yum install git -y",
      "sudo yum install httpd -y"
    ]
  }],
  "post-processors":[{
    "type": "amazon-ami-management",
    "only": ["{{user `post`}}"],
    "region": "ap-southeast-1",
    "identifier": "packer-example",
    "keep_releases": "1"
  }]
}

packer validate template.json

Error:
Error initializing core: 1 error(s) occurred:

• post-processor 1.1: 'only' specified builder '{{user post}}' not found

[ankitkl]

@mitchellh Help required

[rickard-von-essen]

You can't use user variables in only. You have to hardcode amazon-ebs.

[mwhooker]

may be able to extend/fix #4508 to solve this.

I'm thinking we can use the same idea to also fix #2679

[wiadams]

"builders": [
{
"type": "googlecompute",
"name": "{{user distribution}}", (in this case i provide "trusty")

"provisioners": [
{
"type": "ansible",
"except": ["trusty"],

Debug mode enabled. Builds will not be parallelized.
trusty output will be in this color.

So i know the build name is taking effect.
however,

Error initializing core: 1 error(s) occurred:

• provisioner 1: 'except' specified builder 'trusty' not found

Now if i change "name": "{{user distribution}}" to "name": "trusty" in my builder, it works.
But what is the point of requiring this to be a hardcoded string?
I am assuming this is the same issue?

[SwampDragons]

You're right; this is the same issue. It's an artifact of how and when we interpolate variables. Fixing it would require a serious architectural rewrite, so it probably won't happen before Packer 2.0.

[wiadams]

Ok thanks, yeah having the "only" and "except" only work for hardcoded strings is kind of limiting. Basically it would nice to have a way to only run certain provisioners based on some kind of user input. Thanks.

[SwampDragons]

Understood; normally people who need this kind of flexibility wrap their packer run in a json-parser that can format the template how they need.

[derks]

If variables can not be supported, would it be possible to support regex? Our builders are named like the following:

• {{user `target`}}
• {{user `target`}}.vagrant
• {{user `target`}}.vagrant-cloud

Where target is something like ubuntu-16.04. That way we can build/test without Vagrant overhead... and if/when complete if we build with .vagrant-cloud it gets pushed. Sans variables, the following using regex would be sufficient for us:

{
    "post-processors": [
        [
            {
                "keep_input_artifact": true,
                "only": [
                    "$(.*)\.vagrant^"
                ],
                "type": "vagrant"
            }
        ]
    ]
}

[SwampDragons]

No, sorry -- based on the architecture of Packer, regex would be just as hard to support as user variables.

I'm going to mark this with the "config 2.0" flag -- we're in the beginning stages of a push to switch to using HCL2 templates and this may be something we can achieve during that refactor.

[yuliyan5]

Here is what kind of worked for me when using the shell provisioner:
I am setting the builder name with an user variable:
variable
"packer_builder_name": "production",

builder
{ "builders": [ { "name": "{{user packer_builder_name}}", "type": "amazon-ebs", .......................

And after that I use the shell provisioner like this:

{ "type": "shell", "inline": [ "sudo echo Starting builder: $PACKER_BUILD_NAME", "if [ $PACKER_BUILD_NAME = production ] ; then echo running production builder; else echo this is not production, skipping&& exit 0; fi", "echo 11111", "echo 222" ] }

Case 1: the shell script is being executed for production.
Case 2: the shell exits with status 0, so it goes to to next executor.

PACKER_BUILD_NAME
https://packer.io/docs/provisioners/shell.html#packer_build_name

[azr]

Hello there, closing this issue as it should now work to set only/except from a variable using HCL2, the whole array could be a variable:

variable "my_only" {
  default = ["this_builder", "that_builder"]
}

#........

build {

#........

  post-processor {
    only = var.my_only
  }
}

[azr]

Ah wait, this is not possible yet:

In order to make this work we will have to pass the HCL2 eval context into this function first;

packer/hcl2template/types.build.post-processor.go

Lines 25 to 36 in cf514d3

	func (p *Parser) decodePostProcessor(block *hcl.Block) (*PostProcessorBlock, hcl.Diagnostics) {
	var b struct {
	Name string `hcl:"name,optional"`
	Only []string `hcl:"only,optional"`
	Except []string `hcl:"except,optional"`
	KeepInputArtifact *bool `hcl:"keep_input_artifact,optional"`
	Rest hcl.Body `hcl:",remain"`
	}
	diags := gohcl.DecodeBody(block.Body, nil, &b)
	if diags.HasErrors() {
	return nil, diags
	}

[vishalbasra]

Ah wait, this is not possible yet:

In order to make this work we will have to pass the HCL2 eval context into this function first;

packer/hcl2template/types.build.post-processor.go

Lines 25 to 36 in cf514d3

	func (p *Parser) decodePostProcessor(block *hcl.Block) (*PostProcessorBlock, hcl.Diagnostics) {
	var b struct {
	Name string `hcl:"name,optional"`
	Only []string `hcl:"only,optional"`
	Except []string `hcl:"except,optional"`
	KeepInputArtifact *bool `hcl:"keep_input_artifact,optional"`
	Rest hcl.Body `hcl:",remain"`
	}
	diags := gohcl.DecodeBody(block.Body, nil, &b)
	if diags.HasErrors() {
	return nil, diags
	}

Hi!
Is there an example to wrap a post-processor around this context so HCL2 vars may be used?

[vishalbasra]

super awesome!
thankyou so much, I'll be testing this out shortly!

[azr]

Hello there, thanks for trying this out @vishalbasra !! Just for the record, this will only work for HCL2 templates ! 🙂.
Tell us how it goes ! 🙂