Watch doesn't work reliably

[FezVrasta]

Hi, first, thanks for maintaining this plugin!

I'm on 3.0.2 and Node 7, and I have a problem with the watch feature.
This is my karma configuration:

autoWatch: true,
preprocessors: {
    [`${root}/src/**/*.js`]: ['rollup'],
    [`${root}/tests/**/*.js`]: ['rollup'],
},
rollupPreprocessor: {
    moduleName: 'test',
    exports: 'named',
    format: 'umd',
    sourceMap: 'inline',
    plugins: [babel({
        presets: [
            ['es2015', { modules: false }],
            'stage-2',
        ],
    })],
},
files: [
    { pattern:`${root}/src/**/*.js`, included: false, watched: true },
    `${root}/tests/styles/*.css`,
    `${root}/tests/functional/*.js`,
    `${root}/tests/unit/*.js`,
],

Describing how my tests work, the test files import the source files of my library and rollup takes care to compile everything.

The problem is that if I change a file in my tests, the watch task works, if instead I change a file of the source of my library, the bundle isn't updated.

Am I doing something wrong? Sometimes it works, sometimes not... randomly.

[jlmakes]

Issue #3 dealt with a very similar problem, and had a decent solution for 3.0.0...

Not perfect though, due to limitations of Karma's watcher.

Which files are not triggering a new bundle? If you can put together a simple repository demonstrating the problem, I will be able much better equipped to release a fix.

[FezVrasta]

Seems to happen randomly on the same exact files. I'm experiencing it with this project.

This is the config

[jlmakes]

Your popper package is not really a simple/isolated example of the problem.

Does the problem occur with any of the files in your files array, or with specific ones?
Have you looked at the DEBUG_INFO output?

The solution is far from ideal, but it simply stores an array of files listed as dependencies during bundling... and if a file changes, all its dependents are rebundled a well.

[FezVrasta]

It occurs with a lot—maybe all—of the files in the files array.

I have not looked at the debug info, I'll take a look when I have a minute.

[jlmakes]

I'm dropping support for Node 0.12 and releasing [email protected] soon. Since relying on Karma's watcher seems limiting, I'm working on handling watch internally and simply notifying karma when files have changed.

If you have in fact uncovered a bug here, I want to make sure that the new solution solves it. Would you be willing to help me test the new release @FezVrasta?

[FezVrasta]

sure

[jlmakes]

Alright well it'll take a bit though, I'm working on a few open source projects.

[FezVrasta]

no problem, it's just a bit annoying, nothing critical

[menangen]

I too have troubles with Karma and rollup + Vue.js components.

// karma.conf.js
files: [
    'browser/src/tests/global.spec.js',
    {pattern: 'browser/src/components/*.vue', served: false, included: false}
],
preprocessors: {
    'browser/src/tests/global.spec.js': ['rollup']
},

global.spec.js

import Vue from 'vue'

import helloTemplate from 'components/hello.vue'
import vueTemplate from 'components/test.vue'

describe('Vue components', function() {

    it('has a dummy Vue.js test', () => {

        const vm = new Vue({
            el: document.createElement('div'),
            render: (createElement) => createElement(helloTemplate)
        });

        expect(vm.$el.querySelector('h1').textContent).toBe('Welcome to Vue.js App')
    });

    it('has second Vue.js test', () => {

        const vm = new Vue({
            el: document.createElement('div'),
            render: (createElement) => createElement(vueTemplate)
        });

        expect(vm.$el.querySelector('h1').textContent).toBe('hello')
    });
});

 

So, in compiled global.spec.js I have bundled the Vue.js library, Vue components and tests specs... but when I change one of Vue components, rollup does not recompile all dependencies.

I have to write changes in main global.spec.js (not in components) for it to rebundle.

So, we have to know how to rebuild rollup cache on file changes.

In my Karma v1.3.0 server logs on dep. changes:

10 01 2017 19:34:34.179:INFO [watcher]: Changed file "/Users/menangen/WebStormProjects/game-vue-nodejs-site/browser/src/components/test.vue".
10 01 2017 19:34:45.269:INFO [watcher]: Changed file "/Users/menangen/WebStormProjects/game-vue-nodejs-site/browser/src/components/test.vue".
10 01 2017 19:34:56.501:INFO [watcher]: Changed file "/Users/menangen/WebStormProjects/game-vue-nodejs-site/browser/src/components/test.vue".
10 01 2017 19:35:12.729:INFO [watcher]: Changed file "/Users/menangen/WebStormProjects/game-vue-nodejs-site/browser/src/tests/global.spec.js".

On every watcher event it would ideally recompile global.spec.js.

[jlmakes]

Hey @menangen, this is the same issue described in detail here.

The problem is that you are only passing global.spec.js to the rollup preprocessor, which means your other files (that you want to trigger a rebuild) aren't going through the preprocessor... and thus don't trigger anything.

I assume you'll need some plugins like rollup-plugin-vue and rollup-plugin-node-resolve (or rollup-plugin-commonjs) to handle Vue components and include external packages into your rollup bundle... but you'll want to add the source files you want to trigger rebuilds to your preprocessor.

Something like this:

files: [
    'browser/src/tests/global.spec.js',
    {pattern: 'browser/src/components/*.vue', served: false, included: false}
],
preprocessors: {
    'browser/src/tests/global.spec.js': ['rollup'],
    'browser/src/components/*.vue': ['rollup'] // now the preprocessor can see these
},

This way, the preprocessor will be able to track dependents and dependencies for both your test and the source files... so making changes to either will result in recompilation.

I'm working on improving the implementation, but this is exactly what version 3 made possible.

[menangen]

@jlmakes yes, it work but karma runs tests twice. First for changed .vue component and second for .spec.js main test file. If I change test file - it recompile .spec.js fast and once, if change only .vue component to broken code - karma runs 1st iteration with log about changes in .vue file and test show succes(!) pass, and on second iteration log write about changes in main .spec.js file - and now test fails. Very strange...

[jlmakes]

"yes, it work but karma runs tests twice."

This is the exact problem I'm looking to solve. There does not appear to be a way to programmatically add files to the preprocessor, so instead it has to trick Karma into rebundling them by changing their time modified.

As you've pointed out, this means the tests re-run for each file that depends on the changed file. There also seems to be some issues where tests can fail, I assume due to this redundant bundling mid-test.

In my personal use, it's been better having an unreliable watcher than none at all, but...

All this is to say that the Karma watcher needs to be disabled, and the preprocessor needs to own the watch process (just like Webpack does).

[menangen]

@jlmakes ok, I dislike Webpack and will be better use browserify like rebuild as you can test from https://github.com/vuejs-templates/browserify

Also if you on Mac, you can use my testing repo with karma-rollup-preprocessor and Vue building:

diskutil erasevolume HFS+ 'RAM Disk' `hdiutil attach -nomount ram://1000000` && cd /Volumes/RAM\ Disk/ && git clone --depth 1 https://github.com/menangen/game-vue-nodejs-site.git --branch tests --single-branch && cd game-vue-nodejs-site && npm install && node_modules/.bin/karma start

This shell command create RAM disk and clones my repo for playing with rollup + Karma. Also in karma.conf.js need change autoWatch: false to autoWatch: true, than working normal only when change .vue component, test runs nice, but on changing .spec.js test file - no restart Karma tests...

[FezVrasta]

Are there any progress on this issue? Thank you!

[jlmakes]

Hey @FezVrasta, thanks for the reminder there’s still work to be done here!

The documentation is all about how to use Karma, but developing plugins requires reverse engineering existing plugins. So, I’ve been looking deeper into the Node API and wrapping my head around how the Webpack plugin handles their custom file watcher.

I ended up releasing v4, which adds support for custom preprocessors (and drops support for Node 0.12), but this issue will be addressed, just bear with me a bit longer.

[elasim]

Sorry again, Any progress on this issue?

[jlmakes]

Not yet @elasim—I have not forgotten about this project—In fact, I use it every day and intend on improving it, I’ve just been busy the past few months working on another project.

What karma-rollup-preprocessor needs is a re-write using it’s own file watcher, so we can more closely control when files are re-processed (similar to how karma-webpack works).

Pull requests are welcome.

[jlmakes]

Re: @FezVrasta @menangen @elasim @michaelbull
@alexanderby @devoto13 @guilhermehn @mjeanroy

I have greatly improved the file watcher in the latest major release (v5)!

Also, some other important changes have been made, that coincide with the breaking changes released in [email protected]:

• The preprocessor no longer works with Rollup v0.47 and below.
• The preprocessor no longer installs rollup for you (you manage what version you have installed). This is to help developers maintain stable builds, while rollup’s API stabilizes.
• You will need to update both your karma configuration, and all rollup options.

Upgrading Karma to [email protected]

I think the simplest way to get your environment upgraded looks like this:

npm i [email protected] [email protected] --save-dev
rm -rf ./node_modules
npm i

Next, you’ll want to update both your Karma and Rollup config. You can see up-to-date karma.conf.js example in the readme, or examine the commit diff.

(Personally, I also had to update rollup.conf.js for my distribution build process.)

I think that’s it!

[mjeanroy]

@jlmakes Thanks for your awesome work!