Unable to use SDK with ESM bundling in AWS Lambda / Nodejs 14.x

[dreamorosi]

A couple of weeks ago AWS announced support for ESM modules in AWS Lambda for node.js >=14.x runtimes. To quote the announcement:

ECMAScript (ES) modules are a packaging format for JavaScript code, used to publish JavaScript code libraries so they can be imported and re-used in other applications. Until now, Lambda’s Node.js runtimes only supported code using the earlier CommonJS packaging format. With this release, customers can use the ES module format for both their function handler and any code they import.

When trying to bundle a function as ESM module with aws-xray-sdk-node as dependency, simply importing the SDK in AWS Lambda an error is thrown:

{
  "errorType": "Error",
  "errorMessage": "Dynamic require of \"util\" is not supported",
  "trace": [
    "Error: Dynamic require of \"util\" is not supported",
    "    at file:///var/task/index.mjs:13:9",
    "    at lib/node_modules/cls-hooked/context.js (file:///var/task/index.mjs:278:16)",
    "    at __require2 (file:///var/task/index.mjs:16:50)",
    "    at lib/node_modules/aws-xray-sdk-core/dist/lib/context_utils.js (file:///var/task/index.mjs:1701:15)",
    "    at __require2 (file:///var/task/index.mjs:16:50)",
    "    at lib/node_modules/aws-xray-sdk-core/dist/lib/aws-xray.js (file:///var/task/index.mjs:5860:24)",
    "    at __require2 (file:///var/task/index.mjs:16:50)",
    "    at lib/node_modules/aws-xray-sdk-core/dist/lib/index.js (file:///var/task/index.mjs:5950:22)",
    "    at __require2 (file:///var/task/index.mjs:16:50)",
    "    at lib/node_modules/aws-xray-sdk/lib/index.js (file:///var/task/index.mjs:6364:19)"
  ]
}

We have received a bug report from a Customer using @aws-lambda-powertools/tracer, an utility that depends on this SDK.

I was able to reproduce the error using CDK & esbuild with ESM output format enabled.

Below a reproduction example with "aws-cdk-lib": "^2.8.0" & "esbuild": "^0.14.11"

import { Stack, StackProps, Duration } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Tracing, Runtime } from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';

export class DemoStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    new NodejsFunction(this, 'HelloWorldFunction', {
      entry: './lib/main.ts',
      handler: 'handler',
      runtime: Runtime.NODEJS_14_X,
      tracing: Tracing.ACTIVE,
      timeout: Duration.minutes(2),
      bundling: {
        format: OutputFormat.ESM // <-- ENABLING THIS CAUSES THE ISSUE
      }
    });
  }
}

Content of lib/main.ts, function has a single dependency of "aws-xray-sdk": "^3.3.4"

import { getSegment } from 'aws-xray-sdk';

export const handler = () => {

  const segment = getSegment();

  try {
    throw new Error('hello');
  } catch (error) {
    console.log(error);
  }
}

Please don't hesitate to ping me if you need additional details.

[revmischa]

I'm getting this exact issue as well, I would really like it if this was fixed!

[ianwremmel]

Pretty sure I've got the same issue with a different error:

Cannot find package 'aws-xray-sdk-core' imported from /var/task/index.mjs\nDid you mean to import aws-xray-sdk-core/dist/lib/index.js?

I'm bundling my lambda using rollup, but since aws-xray-sdk-core and aws-sdk don't bundle well, I've got them excluded from the bundle and installed in a Lambda Layer. They import fine if I don't use .mjs for my handler.

[NathanielRN]

Hi! Thank you for raising this issue, sorry for the delay in responding.

I did some digging around and read this really useful article on how to build a package for both CommonJS and ESM.

Based on this, I learnt that we need to build the aws-xray-sdk-core package twice, once for both platforms. The current aws-xray-sdk-core package only builds from CommonJS right now as can be seen by how to dist folder is created for the release and by our compilerOptions for when we build the module.

aws-xray-sdk-node/packages/core/package.json

Line 35 in f9c25f4

"compile": "tsc && npm run copy-lib && npm run copy-test",

aws-xray-sdk-node/packages/core/tsconfig.json

Line 3 in f9c25f4

"module": "commonjs",

It would definitely be great to employ something like in that article to have a single module that supports both import and require. This would need us to change our build to

1. Compile for both targets like tsc -p tsconfig-mjs.json && tsc -p tsconfig-cjs.json
2. Have both "type": "commonjs" and "type": "module" types in our package.json files
3. Define an export mapping to route import and require to the right entry point files mjs vs cjs

Because of all of this I'm labeling this issue as an "enhancement" instead of a "bug", please let me know if you think this isn't the best label for this.

I have also created a ticket internally to track this and will bring it up to our team to hopefully get priority on this soon!

They import fine if I don't use .mjs for my handler.

@ianwremmel By handler do you mean the Lambda handler? Also, this is with the latest release of aws-xray-sdk-core? If it is, this sounds like a super useful work around for users 🙂

[ianwremmel]

Because of all of this I'm labeling this issue as an "enhancement" instead of a "bug", please let me know if you think this isn't the best label for this.

This article on the AWS blog generated a lot of buzz a few weeks back, but since the XRay SDK (and full aws-sdk) don't work with it, the article is announcing a feature no one can use, so it does feel more like a bug than an enhancement.

By handler do you mean the Lambda handler? Also, this is with the latest release of aws-xray-sdk-core? If it is, this sounds like a super useful work around for users 🙂

No, sorry, I was saying that if I don't use module mode, everything works, implying I have the layers working correctly. There's no workaround there, I was just trying to get a jump start on potential triage questions you might ask me.