[core] make the asset bundler able to re-use assets

[misterjoshua]

I'd like to request that the asset bundler re-use assets that it has already created if it's instructed to use a source asset hash type.

Use Case

As it stands, the bundler runs the bundling containers before it checks to see if the assets have already been emitted. This is a problem for me when assets are slow to bundle and when there are several stacks (say, one per deployment environment) in a single app. Having the bundler skip already-emitted assets could save a lot of build and test time. It would also tighten the dev loop, which is slowed by unit tests that drive any of the custom bundling assets that invoke Docker.

Proposed Solution

I'd suggest that the bundler containers not be re-run when the source asset hash matches an already-existing asset in cdk.out.

Other

Here's an example project that would benefit from reusing the bundled asset. The stacks run the bundler at least three times to generate the same lambda layer. https://github.com/misterjoshua/aws-cdk-bundle-reuse (And a github workflow)

I've defined this app:

const app = new cdk.App();

new BundlerStack(app, 'BundlerStack-Test');
new BundlerStack(app, 'BundlerStack-Stage');
new BundlerStack(app, 'BundlerStack-Production');

bundler-stack.ts:

export class BundlerStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const pythonDir = path.join(__dirname, 'python');

    const libLayer = new lambda.LayerVersion(this, 'LibLayer', {
      code: lambda.Code.fromAsset(pythonDir, {
        exclude: [ '*', '!requirements.txt' ],
        bundling: {
          image: lambda.Runtime.PYTHON_3_8.bundlingDockerImage,
          command: [ 'bash', '-c', `pip install -r requirements.txt -t /asset-output/python`],
        }
      })
    });

    new lambda.Function(this, 'FunOne', {
      code: lambda.Code.fromAsset(pythonDir, { exclude: [ '.venv' ] }),
      handler: 'lambda_one.handler',
      runtime: lambda.Runtime.PYTHON_3_8,
      layers: [ libLayer ],
    });

    new lambda.Function(this, 'FunTwo', {
      code: lambda.Code.fromAsset(pythonDir, { exclude: [ '.venv' ] }),
      handler: 'lambda_two.handler',
      runtime: lambda.Runtime.PYTHON_3_8,
      layers: [ libLayer ],
    });
  }
}

cdk list takes over a minute for me in this example. The bundler is running three separate pip install containers (one for each stack) even though the results should be identical, based on the source hash. I believe this long time to show a cdk list happens because the bundling occurs during the construction phase, rather than during synthesis.

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[eladb]

Copy @jogold