Yarn support #340

HyperBrain · 2018-03-08T12:56:42Z

What did you implement:

Closes #286

How did you implement it:

Yarn support is provided by an additional packager class that uses the yarn executable instead of npm.
The packager is selected by the webpack: packager setting introduced in serverless-webpack version 5.0.0.

The implementation honors any existing yarn.lock file and provides the same reproducibility as the locked npm packages. It even should behave better, because --frozen-lockfile is used with the yarn install command. Additionally, some special packager options are supported in webpack: packagerOptions:

custom:
  webpack:
    webpackConfig: ./webpack.config.js
    includeModules: true
    packager: yarn  # has to be set explicitly as it defaults to npm
    packagerOptions:
      flat: true   # Will set --flat on install (defaults to false)
      ignoreScripts: true  # Do not execute package.json hook scripts on install (defaults to false)

How can we verify it:

Use Yarn in your project - preferably with a yarn.lock file - and set the packager accordingly in serverless.yml. A serverless package or serverless deploy should do as before, but use yarn and the yarn.lock to create the packages.
You can verify the installation by inspecting the generated zip files in .serverless (best to be done with serverless package.

With Yarn it is not needed to copy the modules explicitly into the function dependencies. For npm copy+prune was faster than install, but yarn is fast as hell here!
With that optimization, the yarn based package/deployment should take less than half of the time that npm does !!! Of course this heavily depends on the machine where it is executed. With a very large file system cache and big amounts of memory, the difference might be rather small.

Your help needed 😄

There are still some uncertainties, especially, if transient dependencies are handled correctly (i.e. if you bundle a first level dependency, its second level dependencies must become first level dependencies). With npm this works as expected and the behavior with yarn should be exactly the same.
Second, we should try this with the most common installation flavors of yarn, from a simple project up to yarn workspaces and shared modules.

Todos:

Unit tests and proper documentation in the README are yet missing. Working on that.

Write tests
Write documentation
Fix linting errors
Make sure code coverage hasn't dropped
Provide verification config / commands / resources
Enable "Allow edits from maintainers" for this PR
Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

HyperBrain · 2018-03-09T10:19:32Z

@franciscocpg This is the promised Yarn support. Can you give it a try and tell me, if it works for you?

Unit tests Added unit tests for copy modules packager flag npm must copy modules Fixed unit tests Yarn does not need to copy modules to be fast Install/prune unit tests. Fixed typo. Copy lock file only if it can be read. AAdjust unit tests. Copy lock file into function directories Only stringify JSON or YML lock files Fixed unit tests Add basic Yarn functionality

RishitKedia · 2018-03-09T10:28:27Z

@HyperBrain Hey, Frank! 👋

Thanks for your time and work on this! You're awesome. ❤️

Just tested this on my project, and I can confirm that it works! 💪

franciscocpg · 2018-03-09T13:59:36Z

@HyperBrain
I'll try ASAP

j0k3r · 2018-03-09T17:11:56Z

❓ I don't know if I have to submit a different issue. If you prefer, I can remove my comment and create a separate issue.

But I just tried on a project and got an error because of flatten deps.

In my package.json I have "colors": "^1.1.2", and in the yarn.lock I have:

[email protected]:
  version "1.0.3"
  resolved "https://registry.yarnpkg.com/colors/-/colors-1.0.3.tgz#0433f44d809680fdeb60ed260f1b0c262e82a40b"

colors@^1.1.2:
  version "1.1.2"
  resolved "https://registry.yarnpkg.com/colors/-/colors-1.1.2.tgz#168a4701756b6a7f51a12ce0c97bfa28c084ed63"

Because winston requires the version 1.0.x:

winston@^2.4.0:
  version "2.4.0"
  resolved "https://registry.yarnpkg.com/winston/-/winston-2.4.0.tgz#808050b93d52661ed9fb6c26b3f0c826708b0aee"
  dependencies:
    async "~1.0.0"
    colors "1.0.x"
    cycle "1.0.x"
    eyes "0.1.x"
    isstream "0.1.x"
    stack-trace "0.0.x"

When running yarn install --frozen-lockfile --non-interactive --flat --ignore-scripts I got:

$ yarn install --frozen-lockfile --non-interactive --flat --ignore-scripts
yarn install v1.5.1
[1/4] 🔍  Resolving packages...
info Unable to find a suitable version for "colors", please choose one by typing one of the numbers below:
  1) "colors@^1.1.2" which resolved to "1.1.2"
  2) "[email protected]" which resolved to "1.0.3"
Answer?:

Which means during the serverless package I got that error:

Error --------------------------------------------------

Command failed: yarn install --frozen-lockfile --non-interactive --flat --ignore-scripts
error An unexpected error occurred: "Can't answer a question unless a user TTY".

You do have --non-interactive so I don't know how can this be fixed.

Using only yarn install doesn't generate that prompt. The prompt arrive when I use --flat.
Which means when flatten deps it needs to chose only one version 😕

HyperBrain · 2018-03-09T17:17:03Z

@j0k3r I'm not sure if that should be the default behavior when flattening as the resolution cannot be determined. We have to use non-interactive mode during packaging.

Do you know if there's a specific resolution switch for yarn, where I can tell to use the latest version, or provide a resolution strategy to yarn?

j0k3r · 2018-03-09T17:23:12Z

@HyperBrain That could be a good option but I've no idea of such option (and I'm not a yarn expert)

HyperBrain · 2018-03-09T17:53:21Z

Ok. Here's a feature request at Yarn: yarnpkg/yarn#1658

It seems that automatic resolution is not yet supported for the flat install option and it is not possible to resolve in non-interactive runs. We should then postpone the flat option support until a solution is available on the Yarn end. Right now I do not see any way to use that in the plugin.
The feature request is quite active recently, so let's hope that it gets done soon. As soon as it is done there we can discuss how to re-add the option here in a meaningful way.

I will remove the option from the PR for now, if everyone agrees, ok?

Gerst20051 · 2018-03-10T01:48:57Z

README.md

+
+#### Yarn
+
+Using yarn will switch the whole packaging pipeline to use yarn, so does it use a `yan.lock` file.


should be yarn.lock not yan.lock

Good catch. Thanks. 👍

HyperBrain · 2018-03-10T10:54:09Z

I removed the flat option as mentioned above for now. We can re-add it as soon as proper Yarn support is there.

j0k3r · 2018-03-11T15:52:03Z

I'll give an other try tomorrow!

SebastianEdwards · 2018-03-11T20:32:04Z

@HyperBrain another Yarn success story here. Works like a charm 👍

j0k3r · 2018-03-12T05:51:43Z

@HyperBrain Works like a charm too.
Really good job 👏

HyperBrain · 2018-03-12T10:37:44Z

Merged.

…support Yarn support

HyperBrain added this to the 5.1.0 milestone Mar 8, 2018

HyperBrain force-pushed the yarn-support branch from 3bf0a0c to bcd8dd6 Compare March 9, 2018 10:26

Frank Schmid added 3 commits March 9, 2018 12:12

Use only Node 6 on appveyor. Test Node 4 on Travis now.

7c34cd5

Unit tests

3b1f163

Updated README

308c20e

Gerst20051 reviewed Mar 10, 2018

View reviewed changes

Frank Schmid added 2 commits March 10, 2018 11:23

Fixed typo

d067aa2

Removed flat option. We have to wait for Yarn to support it.

321f457

HyperBrain merged commit 3176937 into master Mar 12, 2018

HyperBrain deleted the yarn-support branch March 12, 2018 10:46

jamesmbourne pushed a commit to jamesmbourne/serverless-webpack that referenced this pull request Oct 15, 2019

Merge pull request serverless-heaven#340 from serverless-heaven/yarn-…

bbb51d8

…support Yarn support

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yarn support #340

Yarn support #340

HyperBrain commented Mar 8, 2018 •

edited

Loading

HyperBrain commented Mar 9, 2018

RishitKedia commented Mar 9, 2018

franciscocpg commented Mar 9, 2018

j0k3r commented Mar 9, 2018

HyperBrain commented Mar 9, 2018

j0k3r commented Mar 9, 2018

HyperBrain commented Mar 9, 2018 •

edited

Loading

Gerst20051 Mar 10, 2018

HyperBrain Mar 10, 2018

HyperBrain commented Mar 10, 2018

j0k3r commented Mar 11, 2018

SebastianEdwards commented Mar 11, 2018

j0k3r commented Mar 12, 2018

HyperBrain commented Mar 12, 2018


		#### Yarn

		Using yarn will switch the whole packaging pipeline to use yarn, so does it use a `yan.lock` file.

Yarn support #340

Yarn support #340

Conversation

HyperBrain commented Mar 8, 2018 • edited Loading

What did you implement:

How did you implement it:

How can we verify it:

Your help needed 😄

Todos:

HyperBrain commented Mar 9, 2018

RishitKedia commented Mar 9, 2018

franciscocpg commented Mar 9, 2018

j0k3r commented Mar 9, 2018

HyperBrain commented Mar 9, 2018

j0k3r commented Mar 9, 2018

HyperBrain commented Mar 9, 2018 • edited Loading

Gerst20051 Mar 10, 2018

Choose a reason for hiding this comment

HyperBrain Mar 10, 2018

Choose a reason for hiding this comment

HyperBrain commented Mar 10, 2018

j0k3r commented Mar 11, 2018

SebastianEdwards commented Mar 11, 2018

j0k3r commented Mar 12, 2018

HyperBrain commented Mar 12, 2018

HyperBrain commented Mar 8, 2018 •

edited

Loading

HyperBrain commented Mar 9, 2018 •

edited

Loading