AWS Lambda - Serverless Framework - Nestjs - Getting Timed Out

[sedhuait]

What are the steps to reproduce this issue?

I have hosted a REST API with AWS Components like API GATEWAY & LAMBDA using Nestjs and Serverless Framework.

// HoneyBadger init code 
import Honeybadger from '@honeybadger-io/js';
declare global {
  // eslint-disable-next-line no-var
  var Honeybadger: Honeybadger;
}
if (!global.Honeybadger) {
  Honeybadger.configure({
    reportData: true,
    apiKey: process.env.HONEYBADGER_SECRET,
    environment: process.env.NODE_ENV,
  });
  console.log('Loading Honeybadger', !!process.env.HONEYBADGER_SECRET);
  global.Honeybadger = Honeybadger;
}

So to integrate with Honeybadger Lambda Handler I have made the changes below. It had typescript error and I refactored it as follows

After refacorting,

export const handler: Handler = Honeybadger.lambdaHandler(async (event, context) => {
  cachedServer = await bootstrapServer();
  return proxy(cachedServer, event as APIGatewayProxyEvent, context as Context, 'PROMISE').promise;
})

What happens?

This setup works properly in serverless offline start "sls offline start"

But When I deploy this on AWS Lambda, the functions are getting timed out after 30 seconds. Its getting stuck in Honeybadger.lambdaHandler as I see the success response in my middleware.
Kindly check my aws metrics dashboard below. Spikes are during my Honeybadger integration test. After that, I removed it, and it came back to normal.

What were you expecting to happen?

It should have got the proper response.

Any logs, error output, etc?

What versions are you using?

Operating System: … AWS
Package Version: … "@honeybadger-io/js": "3.2.7", "@nestjs/common": "8.0.0", "@nestjs/core": "8.0.0",

[subzero10]

Hey @sedhuait , thanks for submitting an issue. I will try to reproduce your issue and come back to you. FYI, we are already working on improving our lambda handler implementation, so stay tuned!

Thanks!

cc @joshuap

[subzero10]

Hey @sedhuait, I created a nest.js + serverless app and I was able to reproduce your issue:

Task timed out after 6.01 seconds

I then tried our upcoming version of honeybadger-js which includes changes in our AWS lambda handler implementation the timeout error went away.
Can you please give it a try and let me know if it works for you?
Just do:

npm i honeybadger-io/honeybadger-js#aws_lambda_async_#677

Then deploy and test.

Thanks!

[sedhuait]

Sure. Gimme sometime. I'll check and update

[sedhuait]

@subzero10 : I have another quick question.

Honeybadger.notify(error, 'Program Error'); is also not working in this setup. I guess lambda function is getting closed before pushing it to Honeybadger. Could you please confirm?

May be related to #327

[sedhuait]

@subzero10 : I have tried with honeybadger-js#aws_lambda_async_#677 it works for one lambda (REST API LAMBDA with APIGatewayProxyHandler Event) but others (other event handlers) i get errors.

export declare function lambdaHandler(handler: APIGatewayProxyHandler): AsyncHandler;

Why it has to be APIGatewayProxyHandler ? Suppose, if I have plain lambda to handle some other events like aws congnito event, or an event from aws event bridge How should I use it?

error TS2345: Argument of type 'CustomMessageTriggerHandler' is not assignable to parameter of type 'APIGatewayProxyHandler'.
  Types of parameters 'event' and 'event' are incompatible.
    Type 'APIGatewayProxyEvent' is not assignable to type 'CustomMessageTriggerEvent'.
      Type 'APIGatewayProxyEventBase<APIGatewayEventDefaultAuthorizerContext>' is missing the following properties from type 'BaseCustomMessageTriggerEvent<"CustomMessage_Authentication">': request, response, version, region, and 4 more.

[subzero10]

@subzero10 : I have another quick question.

Honeybadger.notify(error, 'Program Error'); is also not working in this setup. I guess lambda function is getting closed before pushing it to Honeybadger. Could you please confirm?

May be related to #327

Yes this is possible.
The Honeybadger.lamdaHandler wrapper actually waits for the exception to be reported before resolving/rejecting the handler.

The tricky part here is that Honeybadger.notify() is built as a fire-and-forget method. That's why we haven't moved forward to make it return a promise. However, for serverless invocations, this might be necessary.
For now, I can suggest a workaround until we decide on this (it should be soon):

async function notifyAsync(error, errorName) {
  return new Promise((resolve, reject) => {
    Honeybadger.notify(error, errorName, {
      afterNotify: function () {
        //optionally, you can do `reject(error)`
        resolve()
      }
    });
  });
}

// call like this
await notifyAsync(error, 'Program Error')

Can you please give it a try and let me know how it goes? Thanks!

cc @joshuap

[subzero10]

@subzero10 : I have tried with honeybadger-js#aws_lambda_async_#677 it works for one lambda (REST API LAMBDA with APIGatewayProxyHandler Event) but others (other event handlers) i get errors.

export declare function lambdaHandler(handler: APIGatewayProxyHandler): AsyncHandler;

Why it has to be APIGatewayProxyHandler ? Suppose, if I have plain lambda to handle some other events like aws congnito event, or an event from aws event bridge How should I use it?

error TS2345: Argument of type 'CustomMessageTriggerHandler' is not assignable to parameter of type 'APIGatewayProxyHandler'.
  Types of parameters 'event' and 'event' are incompatible.
    Type 'APIGatewayProxyEvent' is not assignable to type 'CustomMessageTriggerEvent'.
      Type 'APIGatewayProxyEventBase<APIGatewayEventDefaultAuthorizerContext>' is missing the following properties from type 'BaseCustomMessageTriggerEvent<"CustomMessage_Authentication">': request, response, version, region, and 4 more.

Can you please provide a snippet of code where this is showing the errors? Basically, I'm looking to find all the missing types and make sure they are available (if they should be).

[sedhuait]

@subzero10 Example for CustomMessageTriggerHandler

import { CustomMessageTriggerEvent, CustomMessageTriggerHandler } from 'aws-lambda';

const basehandler: CustomMessageTriggerHandler = async (event: CustomMessageTriggerEvent) => {
  //
  if (event.userPoolId === process.env.AWS_COGNITO_USER_POOL_ID) {
    if (event.triggerSource === 'CustomMessage_ForgotPassword') {
      if (event.request.codeParameter) {
      // some code
      }
    }
  }
  return event;
};

export const handler = Honeybadger.lambdaHandler(basehandler);

Ideally HB should support all the events and handlers from
node_modules/@types/aws-lambda/trigger

[subzero10]

Good point @sedhuait, I am working on this at the moment. I will see if we can have a typed solution or if we will just use unknown, meaning that you will have to cast to the correct type inside your handler.

[subzero10]

@sedhuait The type definitions are done. Honeybadger.lambdaHandler is now a generic function.
The above should work with this:

export const handler = Honeybadger.lambdaHandler<CustomMessageTriggerEvent>(basehandler);

// if you know the result type, you can pass as a second type:
export const handler = Honeybadger.lambdaHandler<CustomMessageTriggerEvent, MyResultType>(basehandler);

Please follow #677 , which should be released soon and will include:

• Fix for the timeout issue
• Honeybadger.notifyAsync
• improved type definitions

cc @joshuap

[subzero10]

This has been released with v4 (currently in beta).