(cdk/pipelines): CFN InternalFailure when provisioning pipeline from CI/CD tutorial

[austinegri]

Describe the bug

I am following the tutorial from https://docs.aws.amazon.com/cdk/v2/guide/cdk_pipeline.html#cdk_pipeline_define.

Bootstrap succeeds - it has admin access.
I run cdk deploy from the above section and I see the following in cloud formation in the PipelineStack events:

pipelinePipeline4163A4B1 | CREATE_FAILED | Internal Failure

I checked cloudtrail for more details on this issue - there is not a single cloudtrail event during the deployment that has an Error Code value - manual inspection of each resource prior to the rollback shows that there don't appear to be issues.

Any advice on how to investigate and fix the root cause of this failure would be appreciated!

Expected Behavior

Successful cdk deploy

Current Behavior

Internal Failure during PipelineStack create

Reproduction Steps

Steps followed from Tutorial: https://docs.aws.amazon.com/cdk/v2/guide/cdk_pipeline.html#cdk_pipeline_bootstrap

1. cdk bootstrap aws://ACCOUNT-NUMBER/REGION --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess
2. Created github project
3. cdk init app --language java
4. In src/main/java/com/myorg/MyPipelineStack.java:

package com.myorg;

import java.util.Arrays;
import software.constructs.Construct;
import software.amazon.awscdk.Stack;
import software.amazon.awscdk.StackProps;
import software.amazon.awscdk.pipelines.CodePipeline;
import software.amazon.awscdk.pipelines.CodePipelineSource;
import software.amazon.awscdk.pipelines.ShellStep;

public class MyPipelineStack extends Stack {
  public MyPipelineStack(final Construct scope, final String id) {
      this(scope, id, null);
  }

  public MyPipelineStack(final Construct scope, final String id, final StackProps props) {
      super(scope, id, props);

      CodePipeline pipeline = CodePipeline.Builder.create(this, "pipeline")
           .pipelineName("MyPipeline")
           .synth(ShellStep.Builder.create("Synth")
              .input(CodePipelineSource.gitHub("austinegri/TmpPipeline2", "mainline"))
              .commands(Arrays.asList("npm install -g aws-cdk", "cdk synth"))
              .build())
           .build();
  }
}

5. In src/main/java/com/myorg/MyPipelineApp.java:

package com.myorg;

import software.amazon.awscdk.App;
import software.amazon.awscdk.Environment;
import software.amazon.awscdk.StackProps;

public class MyPipelineApp {
  public static void main(final String[] args) {
      App app = new App();

      new MyPipelineStack(app, "PipelineStack", StackProps.builder()
          .env(Environment.builder()
              .account("111111111111")
              .region("us-east-1")
              .build())
          .build());

      app.synth();
  }
}

git add --all
git commit -m "initial commit"
git push
cdk deploy

Possible Solution

Unclear

Additional Information/Context

No response

CDK CLI Version

2.115.0 (build 58027ee)

Framework Version

No response

Node.js Version

Welcome to Node.js v20.10.0.

OS

Ubuntu 22.04.3 LTS

Language

Java

Language Version

1.8

Other information

https://github.com/austinegri/TmpPipeline2

[pahud]

Looks like the pipeline was not successfully created by cloudformation due to internal failure.

Some things I would recommend for the next steps to investigate:

1. Can you try the provided sample with CDK in TypeScript and see if it is failing in TypeScript as well?
2. You might need to reach out to AWS premium support to help investigation as this seems to be the codepipeline internal issue.

[austinegri]

Thanks for taking a look @pahud,

1. I attempted performing the same in TS and got the same error: https://github.com/austinegri/TmpPipeline
2. Unfortunately, I do not have AWS premium support. I am just a guy trying to complete a tutorial -

Do you have any further suggestions to investigate?

[pahud]

OK.

input: pipelines.CodePipelineSource.gitHub('austinegri/TmpPipeline', 'mainline'),

This essentially use the GitHub (Version 1) action which is not recommended(details).

Please use the connectionArn instead like:

    const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
      pipelineName: 'MyPipeline',
      synth: new pipelines.ShellStep('Synth', {
        // input: pipelines.CodePipelineSource.gitHub('austinegri/TmpPipeline', 'mainline'),
        input: pipelines.CodePipelineSource.connection('austinegri/TmpPipeline', 'mainline', {
          connectionArn: 'arn:aws:codestar-connections:<REGION>:xxxxxxxxxxxx:connection/xxxxxxxxx',
        }),
        commands: ['npm ci', 'npm run build', 'npx cdk synth']
      })
    });

see: https://github.com/aws/aws-cdk/tree/main/packages/aws-cdk-lib/pipelines#github-github-enterprise-bitbucket-using-a-connection

[austinegri]

Thank you so much @pahud! Using the V2 connection I was able to cdk deploy successfully!

Update - I now see that the troubleshooting section mentions needed a Github access token is needed - but this was not mentioned in the setup steps so I didn't notice this at all