Unable to import module #25

oleschaper · 2017-07-27T09:31:52Z

Hi,

the plugin does not work for me, from CloudWatch Logs:

Unable to import module '_warmup/index': Error at Function.Module._resolveFilename (module.js:469:15) at Function.Module._load (module.js:417:25) at Module.require (module.js:497:17) at require (internal/module.js:20:19)
Unable to import module '_warmup/index': Error
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)

Does it only work if you are using nodejs runtime?

The text was updated successfully, but these errors were encountered:

goncaloneves · 2017-07-27T11:53:39Z

@oleschaper provide more information such as:

sls version
sls plugins you are using in which order
plugin version
plugin options
which command did you run to deploy?
have you confirmed that the warmup lambda is created?

(Edit): No, should work fine with all runtimes. Only the warmup lambda is written in Nodejs.

oleschaper · 2017-07-27T13:00:50Z

Thank you, sorry here are a few more information.

Serverless Version:
1.18

Plugins:

serverless-dynamodb-local, Version: ^0.2.22
serverless-plugin-aws-alerts, Version: ^1.1.0
serverless-plugin-warmup, Version: ^3.0.5-rc.1

I'm using the java8 runtime.
I upload a jar archive that is build using maven to AWS.
The warumup lambda is created, the error log I posted earlier is from the lambda log.

I guess the problem is that the serverless warmup plugin is a runtime dependency and maven does not include any nodejs dependencies?

goncaloneves · 2017-07-29T15:05:33Z

Warmup lambda execution only depends on the available lambda runtime aws-sdk, there is no external dependency as you can check here.

If it's a path issue and you have your warmup created. What is the path of the function handler in the console? Can you invoke your warmup manually?

Let us know your findings.

oleschaper · 2017-08-01T12:51:34Z

Hi,

thank you for your reply!

So the Lambda Function and Log Group is created.
If I try to invoke it manually in the aws lambda web interface, I get the following error message:
{ "errorMessage": "Cannot find module '/var/task/_warmup/index'", "errorType": "Error", "stackTrace": [ "Function.Module._load (module.js:417:25)", "Module.require (module.js:497:17)", "require (internal/module.js:20:19)" ] }

Not sure what you mean by "path of the function handler in the console"?

Thank you for your help!

shahbour · 2017-08-18T07:20:36Z

Hello

Same for me , i am running the project for aws-maven-java and i got same error above

I Received the below WARN when installing the plugin

localhost:spring-cloud-function-adapter-sample shahbour$ npm install serverless-plugin-warmup --save-dev
npm WARN saveError ENOENT: no such file or directory, open '/Users/shahbour/IdeaProjects/spring-cloud-function-adapter-sample/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/Users/shahbour/IdeaProjects/spring-cloud-function-adapter-sample/package.json'
npm WARN spring-cloud-function-adapter-sample No description
npm WARN spring-cloud-function-adapter-sample No repository field.
npm WARN spring-cloud-function-adapter-sample No README data
npm WARN spring-cloud-function-adapter-sample No license field.

+ [email protected]
added 6 packages in 3.791s

deploying was successful

localhost:spring-cloud-function-adapter-sample shahbour$ serverless deploy
Serverless: WarmUP: setting 1 lambdas to be warm
Serverless: WarmUP: aws-java-maven-prod-list-customer
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...............................
Serverless: Stack update finished...
Service Information
service: aws-java-maven
stage: prod
region: us-east-2
api keys:
  None
endpoints:
  GET - https://cby15kvb6f.execute-api.us-east-2.amazonaws.com/prod/customer
functions:
  list-customer: aws-java-maven-prod-list-customer
  warmUpPlugin: aws-java-maven-prod-warmUpPlugin

but then checking the cloudwatch logs it shows

REPORT RequestId: e3447039-83e2-11e7-9c77-3b68224375f9 Duration: 9.15 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 18 MB
START RequestId: e3447039-83e2-11e7-9c77-3b68224375f9 Version: $LATEST
Unable to import module '_warmup/index': Error
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
END RequestId: e3447039-83e2-11e7-9c77-3b68224375f9
REPORT RequestId: e3447039-83e2-11e7-9c77-3b68224375f9 Duration: 20.43 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 18 MB
START RequestId: e3447039-83e2-11e7-9c77-3b68224375f9 Version: $LATEST
Unable to import module '_warmup/index': Error
at require (internal/module.js:20:19)
END RequestId: e3447039-83e2-11e7-9c77-3b68224375f9
REPORT RequestId: e3447039-83e2-11e7-9c77-3b68224375f9 Duration: 9.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 18 MB

yoyeung · 2017-08-22T04:07:04Z

Cannot find module 'bluebird'

yoyeung · 2017-08-22T04:10:51Z

when serverless-webpack go to pack the js file and they need this

/***/ 0:
/***/ (function(module, exports) {

module.exports = require("babel-runtime/core-js/json/stringify");

/***/ }),

/***/ 2:
/***/ (function(module, exports) {

module.exports = require("bluebird");

There is the issue.
No idea with webpack thinking... anyone can help?

yoyeung · 2017-08-22T05:53:17Z

Hi All,
I find a solution. the issue is come from babel-loader.
They try to convert JSON.stringify and promise to Bluebird and babel-runtime.
So what did i do is add this config inside webpack.config.js

new CopyWebpackPlugin([
      { from: './node_modules/bluebird',
        to: '_warmup/node_modules/bluebird'
      },
      { from: './node_modules/babel-runtime',
        to: '_warmup/node_modules/babel-runtime'
      },
      { from: './node_modules/core-js',
        to: '_warmup/node_modules/core-js'
      }])

May be different project have different require. So please check _webpack/index.js

oleschaper · 2017-08-26T11:29:35Z

Hi @yoyeung

Thank you for your answer but I do not understand what you are talking about.
Could you please explain:

Where is "Cannot find module 'bluebird'" coming from?
Where does serverless-webpack put the js files?
Where did you put the module.exports?
Where do I find the webpack.config.js?

Thank you very much.

oleschaper · 2017-08-26T12:06:33Z

I was reading the serverless docs about packaging again (https://serverless.com/framework/docs/providers/aws/guide/packaging#artifact) and it says:

For complete control over the packaging process you can specify your own artifact zip file. Serverless won't zip your service if this is configured and therefore exclude and include will be ignored. Either you use artifact or include / exclude.
The artifact option is especially useful in case your development environment allows you to generate a deployable artifact like Maven does for Java.

So again, my guess would be that the warmup lambda js function/files are just not included in the artifact that is uploaded to aws because it is not in the jar file that maven generates (since it is not a java dependency) and serverless ONLY uploads that artifact file.

Solution would be to include the needed files into the artifact by adding them to the maven pom.xml but I don't know what to include (something called _warmup/index I guess from the logs) and where to find it.

Any help/thoughts?

Thanks!

oleschaper · 2017-08-26T12:25:16Z

Ok, so I found that you are using individual packaging for the warmup function here:
https://github.com/FidelLimited/serverless-plugin-warmup/blob/master/src/index.js#L271-L275

Maybe this is also ignored by serverless if I use an artifact file?

oleschaper · 2017-08-27T20:31:58Z

It is working for me now but the solution is not optimal.
My guess was right, you have to include the warmup function in your jar manually since packaging: individually: is ignored if you are using packaging: artifact: in your service (which I guess everyone using java would do).
Would be great to fix this.
Thanks!

goncaloneves · 2017-08-28T16:24:30Z

Hi @oleschaper.

Thanks for discovering the cause of your issue.

What comes to my mind as a solution is adding this logic within your artifact build logic:

run serverless package command
extract the warmup lambda from the .serverless folder
repackage the warmup lambda inside your artifact

oleschaper · 2017-08-28T17:04:29Z

Hi,

yes that is what I ended up with basically, although I had to modify the plugin in line https://github.com/FidelLimited/serverless-plugin-warmup/blob/master/src/index.js#L154
with this:

return fs.move(this.pathFolder, 'src/main/resources/_warmup', { overwrite: true })

otherwise it would delete the lambda function.
Would be great to include that if some option is set maybe.

oleschaper · 2017-08-28T18:31:07Z

Oh and the order of commands would be:

sls package
mvn package
sls deploy

Not a very nice solution but it works and I don't see any other way for now.
Would be great to introduce an option resourcePath in the plugin options to set the java resource folder and if it is set use
fs.move(this.pathFolder, this.warmup.resourcePath '/' + this.folderName, { overwrite: true })
instead of
return fs.removeAsync(this.pathFolder)

goncaloneves · 2017-08-29T10:17:00Z

@oleschaper get the latest version and use cleanFolder: false.

I am closing this down. 👍

dscis · 2017-10-17T09:10:05Z

So how did you guys resolve the issue? Even with cleanFolder set false the issue still remains.

chaitanya11 · 2018-02-14T12:06:37Z

@gamaro7 i have tried with cleanFolder: false with latest version of this plugin. But this issue still persists. I am doing packaging: artifact. I have tried sls package, but i cannot find warmup lambda function code in .serverless dir.
i could find only some json files.

chaitanya11 · 2018-02-14T13:50:57Z

I got that issue ...
I copied from Readme.md of this project, which contains ',' after cleanFolder: false.

ahharu · 2018-04-09T16:40:45Z

Same boat , ended up ditching this plugin and using https://www.npmjs.com/package/serverless-plugin-lambda-warmup?activeTab=readme . Just works and no PITA

goncaloneves · 2018-04-09T17:39:37Z

The problem was that last release had an issue getting SLS options. I've forgot to merge and push the patch PR version out last month. Done it now @chaitanya11.

Previous versions were working fine.

chaitanya11 · 2018-04-11T03:53:27Z

Thanks @goncaloneves

goncaloneves · 2018-04-11T09:43:33Z

Thank you @chaitanya11 👌

AndrinGautschi · 2018-04-25T13:47:52Z

If anyone other (next to me) is struggling over this issue and has problems solving it with the above discussion:

Issue

When you install the plugin via npm and you run sls package, a directory called '_warmup' within your service root will be created. If you now run sls deploy, the (in serverless.yml) specified artifact (package: artifact: artifactpath) will be uploaded to the deploy bucket on aws. But the _warumup dir which consists of the warump function written in javascript is ignored. This causes the "module not found" error in CloudWatch

The solution

You have to move the _warmup dir into your resources folder (src.main.resources) of your service. This folder will be unpacked by maven and will be added to the root of the .jar. AWS will search within this jar for a dir '_warmup' which consists of a index.js.

In commands

All commands are from the root of your service

sls package

'_warmup' will be created

mv _warmup src/main/resources/

move the folder to your resources dir

mvn clean package

package the artifact

sls deploy

deploys the jar

and here is my serverless.yml conf for the plugin

custom:
  ...
  warmup:
    memorySize: 256
    schedule: 'cron(0/5 8-17 ? * MON-FRI *)' # Run WarmUP every 5 minutes Mon-Fri between 8:00am and 5:55pm (UTC)
    timeout: 20
    cleanFolder: false
    prewarm: true # Run WarmUp immediately after a deployment
    tags:
      Project: **
      Owner: **

goncaloneves · 2018-04-25T14:18:10Z

@AndrinGautschi Thanks for the clear info of what is happening. Just to confirm you are using the latest version 3.6.1?

Ok, I guess it took me a "while" to understand the issue here. I don't do manual artifacts on my deploy pipelines. Basically when defining a custom Serverless artifact package path the _warmup folder is still under the main service folder, making it not being present in the zip file that get's deployed.

So solution for this is fetch the artifact path when is defined and have the _warmup folder in there instead.

AndrinGautschi · 2018-04-25T16:34:43Z

@goncaloneves Thanks for the fast reaction.

Yes, I use the latest version.

Yes that would do it I suppose. Another solution would be to make another configuration parameter to specify where the "_warmup" folder should be placed. Then we could specify by ourself. With a little doc update, this would solve the issue (hacky, but efficient).

goncaloneves · 2018-04-26T13:37:38Z

@AndrinGautschi Thanks for the reply. 👌 Having both options probably is the ideal solution, default sets root service path, artifact path if set, or plugin configurable path. That will tick all boxes. Hacking like pros hehe

I will push this forward when I work on the plugin.

jvgeee · 2018-05-02T05:33:23Z

@goncaloneves @AndrinGautschi I'm not using Maven (and my deployment happens via Jenkins), plus my directory structure seems to be a different to you guys':

.
├── resource
│   └── // database initialization, swagger documentation tooling, authentication...
├── src
│   ├── service
│   │   └── profile
│   │       ├── handler.js
│   │       ├── profile.fn.yml
       └── // All services have their own folders in here

When you say You have to move the _warmup dir into your resources folder (src.main.resources) of your service, is this a specific /resources folder called resource within each service's directory?

Like, should I make my structure like this:

├── src
│   ├── service
│   │   └── profile
│   │   │   └── resource
│   │   │   │   └── _warmup
│   │   │   │       ├── index.js

│   │   └── search
│   │   │   └── resource
│   │   │   │   └── _warmup
│   │   │   │       ├── index.js

And so on and so forth?

EDIT: I've done this, but still getting:

Unable to import module '_warmup/index': Error
at __webpack_require__ (/var/task/_warmup/index.js:20:30)
at Object.30 (/var/task/_warmup/index.js:78:1)
at __webpack_require__ (/var/task/_warmup/index.js:20:30)

For reference:

in serverless.yml I have package: individually: true and warmup: cleanFolder: false
My webpack.config.js looks like this:

var yaml = require('js-yaml')
var fs = require('fs')
var path = require('path')
var nodeExternals = require('webpack-node-externals')
var slsw = require('serverless-webpack')

module.exports = {
  entry: slsw.lib.entries,
  // Generate sourcemaps for proper error messages
  devtool: 'source-map',

  // webpack-node-externals
  // don't bundle built in modules like 'fs', 'path' etc.
  target: 'node',

  // webpack-node-externals
  // Define 'externals' which won't be included in the bundle that webpack generates.
  // We're excluding everything in node_modules.
  externals: [nodeExternals()],

  // Run babel on all .js files and skip those in node_modules
  module: {
    rules: [{
      test: /\.js$/,
      loader: 'babel-loader',
      include: __dirname,
      exclude: /node_modules/
    }]
  },
  output: {
    libraryTarget: 'commonjs',
    path: path.join(__dirname, '.webpack'),
    filename: '[name].js',
    devtoolModuleFilenameTemplate: '[absolute-resource-path]',
    devtoolFallbackModuleFilenameTemplate: '[absolute-resource-path]?[hash]'
  }
};

juanjoDiaz · 2018-10-06T23:00:59Z

There seem to be multiple issues going on here.
It would be good to create separate issues.

When packaging using package: artifact (Maven) projects, _warmup is not included.
As already established, the solution is to add an option to customize the path where the warmup files are stored.

When using serverless-webpack, it can't find _warmup
Is the warmup plugin declared before the webback one? Otherwise, webpack is run before warmup has generated its files.
Anyway, why do you need to process the warmup handler using webpack?
You config says // Run babel on all .js files and skip those in node_modules
Probably excluding anything inside _warmup is the solution...

juanjoDiaz · 2018-10-07T08:12:59Z

Thinking a bit more about this...

Would it be a solution to simply apply the package: artifact: at the function level instead of the service level?

So, instead of

package:
  artifacts: path/to/my-artifact.zip

functions:
  myFunction:
    ...

use

functions:
  myFunction:
    ...
    package:
      artifacts: path/to/my-artifact.zip

That way, serverless still packages the lambdas individually and correctly generates the package for the warmup plugin.

juanjoDiaz · 2018-10-07T11:11:50Z

Actually, just created a PR in Serverless that fix the root cause of this issue (the individually config is not honored when there is a package: artifact: ... set up at service level).

Let's see if it can get merge and we won't need to workaround the issue 🙂

juanjoDiaz · 2018-10-15T08:00:50Z

Alrighty! My PR is merged into Serverless.
So the next release of serverless you be able to solve the issues mentions in this issue.

Essentially, now you can do:

service: test

provider:
  name: aws
  runtime: nodejs8.10

package:
  artifact: my-app.zip # This can be an empty zip

functions:
  normalFunc:
    handler: func/handler.func
    events:
      - http:
          path: func
          method: post
  individualFunc:
    handler: func/handler.func
    events:
      - http:
          path: func2
          method: post
    package:
       individually: true

So running serverless package with the above config will use my-app.zip as artifact for all functions but will create a zip for the ones set as package: individually: true like func/handler.func (the func folder needs to exist of course).

Since serverless-plugin-warmup is set as package: individually: true, it will always get its own zip.

psiphi75 · 2018-10-16T22:17:54Z

EDIT: It works now for me. Flushing my node_modules and reinstalling seemed to do the trick.

~~I'm also getting the same/similar error message. I'm using serverless v1.32.0 and nodejs8.10 as the runtime.~~

My relevant serverless config is:

plugins:
  - serverless-plugin-warmup # I have tried this at the end of the list
  - serverless-webpack
  - serverless-plugin-tracing
  - serverless-mocha-plugin

custom:
  warmup:
    default: false # We enable this per function
    folderName: _warmup
    schedule: rate(5 minutes)
    cleanFolder: false
    prewarm: true 

package:
  individually: true

functions:
  myFunc:
    handler: src/myFunc/handler.handler
    warmup: true
    events:
      ...

The error message is:

Unable to import module 'src/myFunc/handler': Error
at require (internal/module.js:11:18)
at Object.<anonymous> (/var/task/node_modules/request/index.js:17:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function._load (/var/task/node_modules/@sentry/node/dist/integrations/console.js:36:47)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at /var/task/node_modules/request-promise-any/lib/rp.js:9:12
at module.exports (/var/task/node_modules/stealthy-require/lib/index.js:62:23)

goncaloneves · 2018-11-15T08:15:02Z

Perfect @juanjoDiaz. Once again, terrific work. 🙌

goncaloneves added the help wanted label Jul 29, 2017

goncaloneves closed this as completed Aug 29, 2017

dscis mentioned this issue Oct 17, 2017

Unable to import module, serverless-webpack #36

Closed

goncaloneves reopened this Apr 25, 2018

goncaloneves added the bug label Apr 25, 2018

richarddd mentioned this issue Oct 5, 2018

Always use a dedicated artifact for warmup function #69

Closed

This was referenced Oct 7, 2018

Circular dependency when using package: artifact #70

Closed

Let function package.individually config override service artifact serverless/serverless#5364

Merged

goncaloneves closed this as completed Nov 15, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to import module #25

Unable to import module #25

oleschaper commented Jul 27, 2017

goncaloneves commented Jul 27, 2017 •

edited

Loading

oleschaper commented Jul 27, 2017 •

edited

Loading

goncaloneves commented Jul 29, 2017

oleschaper commented Aug 1, 2017

shahbour commented Aug 18, 2017

yoyeung commented Aug 22, 2017

yoyeung commented Aug 22, 2017

yoyeung commented Aug 22, 2017

oleschaper commented Aug 26, 2017

oleschaper commented Aug 26, 2017 •

edited

Loading

oleschaper commented Aug 26, 2017

oleschaper commented Aug 27, 2017

goncaloneves commented Aug 28, 2017

oleschaper commented Aug 28, 2017

oleschaper commented Aug 28, 2017

goncaloneves commented Aug 29, 2017

dscis commented Oct 17, 2017

chaitanya11 commented Feb 14, 2018

chaitanya11 commented Feb 14, 2018

ahharu commented Apr 9, 2018

goncaloneves commented Apr 9, 2018

chaitanya11 commented Apr 11, 2018

goncaloneves commented Apr 11, 2018

AndrinGautschi commented Apr 25, 2018 •

edited

Loading

goncaloneves commented Apr 25, 2018

AndrinGautschi commented Apr 25, 2018

goncaloneves commented Apr 26, 2018

jvgeee commented May 2, 2018 •

edited

Loading

juanjoDiaz commented Oct 6, 2018

juanjoDiaz commented Oct 7, 2018 •

edited

Loading

juanjoDiaz commented Oct 7, 2018

juanjoDiaz commented Oct 15, 2018

psiphi75 commented Oct 16, 2018 •

edited

Loading

goncaloneves commented Nov 15, 2018

Unable to import module #25

Unable to import module #25

Comments

oleschaper commented Jul 27, 2017

goncaloneves commented Jul 27, 2017 • edited Loading

oleschaper commented Jul 27, 2017 • edited Loading

goncaloneves commented Jul 29, 2017

oleschaper commented Aug 1, 2017

shahbour commented Aug 18, 2017

yoyeung commented Aug 22, 2017

yoyeung commented Aug 22, 2017

yoyeung commented Aug 22, 2017

oleschaper commented Aug 26, 2017

oleschaper commented Aug 26, 2017 • edited Loading

oleschaper commented Aug 26, 2017

oleschaper commented Aug 27, 2017

goncaloneves commented Aug 28, 2017

oleschaper commented Aug 28, 2017

oleschaper commented Aug 28, 2017

goncaloneves commented Aug 29, 2017

dscis commented Oct 17, 2017

chaitanya11 commented Feb 14, 2018

chaitanya11 commented Feb 14, 2018

ahharu commented Apr 9, 2018

goncaloneves commented Apr 9, 2018

chaitanya11 commented Apr 11, 2018

goncaloneves commented Apr 11, 2018

AndrinGautschi commented Apr 25, 2018 • edited Loading

Issue

The solution

In commands

goncaloneves commented Apr 25, 2018

AndrinGautschi commented Apr 25, 2018

goncaloneves commented Apr 26, 2018

jvgeee commented May 2, 2018 • edited Loading

juanjoDiaz commented Oct 6, 2018

juanjoDiaz commented Oct 7, 2018 • edited Loading

juanjoDiaz commented Oct 7, 2018

juanjoDiaz commented Oct 15, 2018

psiphi75 commented Oct 16, 2018 • edited Loading

goncaloneves commented Nov 15, 2018

goncaloneves commented Jul 27, 2017 •

edited

Loading

oleschaper commented Jul 27, 2017 •

edited

Loading

oleschaper commented Aug 26, 2017 •

edited

Loading

AndrinGautschi commented Apr 25, 2018 •

edited

Loading

jvgeee commented May 2, 2018 •

edited

Loading

juanjoDiaz commented Oct 7, 2018 •

edited

Loading

psiphi75 commented Oct 16, 2018 •

edited

Loading