Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module initialization error #1287

Closed
pputhran opened this issue May 5, 2016 · 4 comments
Closed

module initialization error #1287

pputhran opened this issue May 5, 2016 · 4 comments
Assignees
Labels
api: bigquery Issues related to the BigQuery API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@pputhran
Copy link

pputhran commented May 5, 2016

Hello,

I am trying to use AWS lambda function to read off Kinesis stream and insert into Google Big Query.
It appears to work fine locally (on my mac) but when I deploy this to AWS and execute as AWS lambda function, I run into initialisation error

error stack:

module initialization error: Error at Error (native)
  at Object.Module._extensions..node (module.js:434:18)
  at Module.load (module.js:343:32)
  at Function.Module._load (module.js:300:12)
  at Module.require (module.js:353:17)
  at require (internal/module.js:12:17)
  at Object.<anonymous> (/var/task/node_modules/gcloud/node_modules/grpc/src/node/src/grpc_extension.js:40:15)
  at Module._compile (module.js:409:26)
  at Object.Module._extensions..js (module.js:416:10)
  at Module.load (module.js:343:32)

Appears to be failing to invoke this
gcloud/node_modules/grpc/src/node/extension_binary/grpc_node.node

Below is the sample code

Sample code:

var gcloud = require('gcloud');

module.exports.handler = function(event, context, cb) {

 var bigquery = gcloud.bigquery({
    projectId: process.env.GCLOUD_PROJECT_ID,
    keyFilename: './keys.json'
  });

  event.Records.forEach(function(record) {
    // Kinesis data is base64 encoded so decode here
    var payload = new Buffer(record.kinesis.data, 'base64').toString('ascii');
    console.log('Decoded payload:', payload);
  });
  cb(null, "message");
};
@jgeewax jgeewax added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. api: bigquery Issues related to the BigQuery API. labels May 5, 2016
@jgeewax
Copy link
Contributor

jgeewax commented May 5, 2016

This very well could be an issue with gRPC not being able to run on Lambda (it uses a pre-compiled binary and needs some fancy support).

@stephenplusplus : Is there a way to wrap the gRPC requirement with a try/except since it really should be an optional module?

Seems we should fail gracefully if gRPC doesn't import correctly (ie, start the change here https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/lib/pubsub/index.js#L32) ?

@stephenplusplus
Copy link
Contributor

Is there a way to wrap the gRPC requirement with a try/except since it really should be an optional module?

We haven't designed it as an optional dependency thus far, i.e. we don't fallback to HTTP via the request module. Once we added support for gRPC to an API, we removed the old stuff. If that's how it should be, we would declare gRPC as an optionalDependency, so that if it fails to install / be require'd (try/catch like you said), we would fall back to request. Should we go in that direction or try to make gRPC compatible with the Lambda environment?

@murgatroid99
Copy link

From the Lambda documentation, it looks like it shouldn't be too hard to get Node gRPC working on Lambda. In fact, I'm not really sure why it doesn't already work. I can look into this, but maybe not immediately.

@stephenplusplus
Copy link
Contributor

@pputhran thanks for reporting. You can track this over in grpc/grpc#6443.

@jgeewax if you'd like us to have a fallback in place if gRPC cannot be installed, please open a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

4 participants