Regression - 6.33.0 AWS Batch job definition - 'Job definition name is required'

[Raggey]

What happened?

Upgraded to latest version and getting an error when updating a batch job definition

Example

aws:batch:JobDefinition (dev-clustering-clustering-job-defn): error: 1 error occurred: * updating urn:pulumi:dev::clustering::aws:batch/jobDefinition:JobDefinition::dev-clustering-clustering-job-defn: 1 error occurred: * deleting Batch Job Definition (): ClientException: Error executing request, Exception : Job definition name is required.

Output of pulumi about

`CLI
Version 3.114.0
Go Version go1.22.2
Go Compiler gc

Plugins
KIND NAME VERSION
resource aws 6.33.0
resource docker 4.5.0
language python unknown

Host
OS ubuntu
Version 22.04
Arch x86_64
`

Additional context

Reverting to 6.32.0 resolves the issue, tested on 6.32.0, 3.13.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

Hi @Raggey thanks for posting this and I'm sorry you're hitting this regression. Could you add a quick sample program that configures JobDefinition and hits the issue? Much appreciated.

[Raggey]

Hi @Raggey thanks for posting this and I'm sorry you're hitting this regression. Could you add a quick sample program that configures JobDefinition and hits the issue? Much appreciated.

Hi @t0yv0 ,

Have managed to narrow down exactly how to re-create it. (Sorry should have posted this in the original issue)

Starting with the example from the docs:

import pulumi_aws as aws

test = aws.batch.JobDefinition(
    "test",
    name="my_test_batch_job_definition",
    type="container",
    container_properties=json.dumps(
        {
            "command": [
                "ls",
                "-la",
            ],
            "image": "busybox",
            "resourceRequirements": [
                {
                    "type": "VCPU",
                    "value": "1",
                },
                {
                    "type": "MEMORY",
                    "value": "512",
                },
            ],
            "volumes": [
                {
                    "host": {
                        "sourcePath": "/tmp",
                    },
                    "name": "tmp",
                }
            ],
            "environment": [
                {
                    "name": "VARNAME",
                    "value": "VARVAL",
                }
            ],
            "mountPoints": [
                {
                    "sourceVolume": "tmp",
                    "containerPath": "/tmp",
                    "readOnly": False,
                }
            ],
            "ulimits": [
                {
                    "hardLimit": 1024,
                    "name": "nofile",
                    "softLimit": 1024,
                }
            ],
        }
    ),
)

We deploy this okay.

Then updating something in the container_properties specifically (I'm updating the container image here:)

import pulumi_aws as aws

test = aws.batch.JobDefinition(
    "test",
    name="my_test_batch_job_definition",
    type="container",
    container_properties=json.dumps(
        {
            "command": [
                "ls",
                "-la",
            ],
            "image": "busybox2",
            "resourceRequirements": [
                {
                    "type": "VCPU",
                    "value": "1",
                },
                {
                    "type": "MEMORY",
                    "value": "512",
                },
            ],
            "volumes": [
                {
                    "host": {
                        "sourcePath": "/tmp",
                    },
                    "name": "tmp",
                }
            ],
            "environment": [
                {
                    "name": "VARNAME",
                    "value": "VARVAL",
                }
            ],
            "mountPoints": [
                {
                    "sourceVolume": "tmp",
                    "containerPath": "/tmp",
                    "readOnly": False,
                }
            ],
            "ulimits": [
                {
                    "hardLimit": 1024,
                    "name": "nofile",
                    "softLimit": 1024,
                }
            ],
        }
    ),
)

Then replicates the issue:

  pulumi:pulumi:Stack (XXX-dev):
    error: update failed

  aws:batch:JobDefinition (test):
    error: 1 error occurred:
        * updating urn:pulumi:XXX/jobDefinition:JobDefinition::test: 1 error occurred:
        * deleting Batch Job Definition (): ClientException: Error executing request, Exception : Job definition name is required.,````

[t0yv0]

Hmm. I can repro this on the latest version. What is even more interesting is that after this change subsequent pulumi up is cycling:

pulumi up --yes # ok
pulumi up --yes # ClientException
pulumi up --yes # ok
pulumi up --yes # ClientException
pulumi up --yes # ok
pulumi up --yes # ClientException

[t0yv0]

It appears there was a recent change that's likely projecting incorrectly to Pulumi hashicorp/terraform-provider-aws#37111

[t0yv0]

This appears to start working correctly in Pulumi after this resource is enrolled in the PlanResourceChange flag; as it fixes the ordering of diff customizer application that I think aligns it with TF behavior. Fix rolling shortly.