Locally invoked functions clear webpack compiled code needed for functions which are called from locally invoked function

[todda00]

This is a Bug Report

Description

When invoking a local function which publishes to an SNS topic, the called lambda function which is subscribed to the SNS topic fails due to the webpack compiled code being removed during the local invocation. This issue is only during local development, and using serverless-offline-sns.

Example repository to highlight the issue: https://github.com/todda00/serverless-offline-sns-example/tree/webpack-local-invoke-issue

Steps to reproduce:

Simply run

npm install
npm start

And then go to http://localhost:3000/ping to trigger the event, you should see "pong" appear in the console.

To see the issue with webpack, open a new command line (while serverless offline is still running) and run:

serverless invoke local -f ping

You will see the trace error in the terminal where serverless offline is running.

The .webpack folder is cleared out during validation and is only compiling the locally invoked function (ping), but the running offline instance needs the pong handler, which has been removed.

For bug reports:

• What went wrong?
  compiled modules of other functions are removed during local invocations
• What did you expect should have happened?
  both functions to be able to be ran in sequence without webpack removing the compiled code
• What was the config you used?
  see example repository: https://github.com/todda00/serverless-offline-sns-example/tree/webpack-local-invoke-issue
• What stacktrace or error message from your provider did you see?

Error: Cannot find module '/Users/toddhumphrey/Repos/serverless-offline-sns-example/.webpack/service/pong'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at ServerlessOfflineSns.createHandler (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/serverless-offline-sns/dist/index.js:199:23)
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/serverless-offline-sns/dist/index.js:178:107
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/serverless-offline-sns/dist/sns-adapter.js:141:29
    at Layer.handle [as handle_request] (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/layer.js:95:5)
    at next (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/layer.js:95:5)
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:335:12)
    at next (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:275:10)
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/serverless-offline-sns/dist/sns-server.js:44:13
    at Layer.handle [as handle_request] (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:317:13)
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:335:12)
    at next (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:275:10)
    at urlencodedParser (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/body-parser/lib/types/urlencoded.js:82:7)
    at Layer.handle [as handle_request] (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:317:13)
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:335:12)
    at next (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/express/lib/router/index.js:275:10)
    at /Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/body-parser/lib/read.js:130:5
    at invokeCallback (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/raw-body/index.js:224:16)
    at done (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/raw-body/index.js:213:7)
    at IncomingMessage.onEnd (/Users/toddhumphrey/Repos/serverless-offline-sns-example/node_modules/raw-body/index.js:273:7)
    at emitNone (events.js:86:13)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)

Similar or dependent issue(s):

Additional Data

• Serverless-Webpack Version you're using: 4.0.0
• Webpack version you're using: 3.8.1
• Serverless Framework Version you're using: 1.24.1
• Operating System: MacOS Sierra v 10.12.6
• Stack Trace (if available):
  see trace above

[HyperBrain]

Hi @todda00 , thanks for reporting 👍
This is indeed a somehow unexpected behavior. With the standard serverless-offline the APIG emulations work and also keep the .webpack folder. However I never tested with the sns plugin. I will check with the repo you provided and see if we're missing a hook here, or if the sns plugin uses a special order here.

[HyperBrain]

I have an idea to solve the problem. If you run serverless-offline the .webpack folder is already set up and watched by serverless-offline. The problem is now, if you run serverless invoke local to start a second function independently, it will use the .webpack folder too (and will clean it before it starts to have no remnants in there).

We could introduce a switch for invoke local like --webpack-use-existing or --webpack-no-clear, that turns off the removal in the validation step. This will allow to run serverless-offline while invoking a second function with the same build. It even could omit the compile completely, as serverless-offline already triggers a recompile if any sources change. So it should solve your problem.

@todda00 What do you think of this approach?

[todda00]

yes, a command line option for invoke local would work great for this situation, I was looking for something available when I ran into the issue, but there was nothing.

[HyperBrain]

I'll add the switch then for these use cases that allows executing a function out of a currently running offline compile. Please let me know if you have any better idea for the switch name - I'm not good in finding names 😄 . The only requirement is, that it should be prefixed with --webpack for disambiguation.

[todda00]

If it is not compiling at all, then perhaps --webpack-no-compile would be applicable.

…

On Tue, Nov 14, 2017 at 5:26 PM, Frank Schmid ***@***.***> wrote: I'll add the switch then for these use cases that allows executing a function out of a currently running offline compile. Please let me know if you have any better idea for the switch name - I'm not good in finding names 😄 . The only requirement is, that it should be prefixed with --webpack for disambiguation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#275 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH-flNZr6KAX7ffwQxTxyAsqklZQksAlks5s2hODgaJpZM4QeD86> .

[HyperBrain]

@todda00 --no-build is now available for invoke local in PR #347 . This should solve the issue. Can you give the PR a try and report back if it is solved?

[HyperBrain]

Released with 5.1.0