click() helper is not reliable

[izelnakri]

// in my new ember acceptance tests

await click('[data-test-username-button]');

assert.equal(currentURL(), '/izelnakri'); // this doesn't pass!

click helper doesnt wait for url transition. It could be either due to fact that ember click selector ignore attribute selectors [data-test-username-button] or its waiter is simply buggy.

Unfortunately I already removed a branch in my project where I was refactoring my application tests one by one to new ember application test/qunit modules. I succeeded with unit + integration tests but failed on application tests.

[Turbo87]

@izelnakri how can we reproduce this? I haven't observed any such issues so far with the new APIs. Can you reproduce it on a fresh ember new app?

[izelnakri]

I just created a dummy app with a simple link click, it does follow and wait for the new link properly. However I have another proprietary app where click() helper ends earlier than it should, not properly waiting for a full route transition.

I've rewritten the test case again from scratch at this point Im 100% certain that this is a bug related to the new click helper. Im 100% certain because even fastboot servers wait for complete promise resolution, doing the full route transition.

Target links route model hook is bit complicated, so Im sharing an example of it here, maybe this could help:

// test code in my new ember acceptance tests

await click('[data-test-username-button]');

assert.equal(currentURL(), '/izelnakri'); // this doesn't pass! click handler ends early.

import Route from '@ember/routing/route';
import { Promise } from 'rsvp';
import { run } from '@ember/runloop';
import ENV from 'frontend/config/environment';

// I've obfuscated the model names and checks however the promise returns are same.

export default Route.extend({
  model({ slug }) {
    if (slug.length > 15) {
      return new Promise((resolve) => {
        return fetch(`${ENV.APP.API_HOST}/slugs/${slug.toLowerCase()}`) // I've changed the link here 
          .then((response) => response.json())
          .then((json) => {
            const keyName = Object.keys(json)[0];

            run(() => this.get('store').pushPayload(json));
            if (keyName === 'blog_post') {
              const blogPost = this.get('store').peekRecord('blog-post', json[keyName].id);

              resolve(this.transitionTo('blog-posts.post', blogPost));
            } else if (keyName === 'organization') {
              const organization = this.get('store').peekRecord('organization', json[keyName].uuid);

              resolve({ type: 'organization', organization: organization });
            }
          });
        });
    }

    return this.get('store').queryRecord('user', { username: slug }).then((user) => {
      return { type: 'user', user: user };
    }).catch(() => this.transitionTo('not-found', slug));
  }
});

[izelnakri]

I figured click() helper finishes earlier due to this code in the model hook:

run(() => this.get('store').pushPayload(json));

when I remove the run loop there test passes. However this application test was passing before I converted my tests to ember v3 testing structure and I've not received any warnings and the solution was very implicit, not ideal. This is bit strange I think that click() helper just waits for the next runloop rather than a complete transition, this should change I think to the previous behavior. It will also make the tests more backward compatible. Speed gains we currently get should be extremely minimal and it introduces a footgun to the developers.

[rwjblue]

It waits for settledness (all scheduled run loops are completed, all test waiters are completed, all pending $.ajax are completed).

What Ember version are you working with? Now that you better understand the issue can you make a reproduction? What were you actually testing (that the items pushed are rendered, some other side effect, etc)?

[izelnakri]

Hi, this occured in version 3.0.0(cli, source, data). I tried to click on a button and assert that url changes. During that the model hook/transition above executes. Model hook above returns a promise(so the templates wait for the initial render, I did this on purpose to get the html for serverside rendering).

I dont know how to reproduce this other than the code above. Maybe we can just write a test case, in it a model hook returns a promise after click() and see if click handler finishes early or on time. Maybe the issue is model hook & templates waits for simple js promises but click test handler doesnt wait for complete transitions but I'm just guessing of course.

[rwjblue]

Ya, once we isolate exactly why the above wouldn’t wait we can figure out if/how to fix...

My gut feeling here was that it should have worked, so I’m surprised that it didn’t

[Turbo87]

@izelnakri have you been able to isolate your issue into a reproduction yet? with that this is pretty much impossible for us to fix.

[izelnakri]

The issue is that it is indeed hard to write a test case for this, as the core issue is ambiguous and my issues occur on private projects which I cannot completely share. I'm pretty sure the new click() helper is less stable then the previous one, had another unexpected issue with it few days ago while continuing with my acceptance test rewrites. The current click helper require non-obvious code changes and not completely backward compatible, I'm 100% sure about that. Unfortunately, don't know the exact reason why it cant properly wait for transitions sometimes, can't help with that for the moment being :(

[walter]

An additional data point about click being unreliable. Mine is a tad different. It's somehow causes more input events than the old click.

Unfortunately I also can't share my code, but I give a rundown of what's happening. I'm testing a set of code that has to do with pin number registration.

There is some jQuery heavy input entry monitoring that triggers focus change, validation, and submission, and reset for confirmation re-entry. On top of that there is custom masking of the numbers entered after focus change. This stuff is old and brittle and hopefully we can replace it with something that relies on standard HTML password field handling in the future.

After the first entry of the 4 digit pin the inputs are cleared and if the user has re-entered the same pin, they can click a newly available button to submit the pin. However, if there is any issue with the pin at this stage, the button is replaced with an error message that describes how to fix the problem.

The old style testing worked as expected against it, though there was a lot of andThen use to get the form to the expected state.

The new problematic testing code looks like this:

    await fillIn('input#pinDigit1', '1');
    await fillIn('input#pinDigit2', '3');
    await fillIn('input#pinDigit3', '2');
    await fillIn('input#pinDigit4', '4');

    await waitFor('button#verify-pin-button');

    assert.step('Filling in second PIN set');
    assert.dom('.Message--alert').doesNotExist();

    await fillIn('input#pinDigit1', '1');
    await fillIn('input#pinDigit2', '3');
    await fillIn('input#pinDigit3', '2');
    await fillIn('input#pinDigit4', '4');

    await click('button#verify-pin-button');    

   // further assertions of successful transition to post-registration

Everything works as expected up to the await click('button#verify-pin-button');. If I leave that in place, the click triggers reset of inputs and then additional pin number input and validation fails.

However, if I pauseTest() before the click and push the button via the browser, the functionality behaves as expected.

Noticing that pin code is overly coupled to specific events being triggered, I messed around with using triggerEvent and triggerKeyEvent rather than click. I couldn't find the right mix though.

Note I was using 0.7.20 and also tried against master.

[mdbiscan]

I am having the same issue. This is from a dummy app for an addon library. It fails in the most basic of tests, but only about 30-50% of the time (both locally and in Jenkins), so it's a bit unreliable.

This was tested with Ember v2.18.2 and v3.1.3.

test('docs route can be visited', async function(assert) {
    await visit('/');
    await click('.link-to-docs');

    assert.equal(currentURL(), '/docs', 'assert docs can be visited from root');
  });

[raphaelns-developer]

I'm having the same issue. I did a little example using ember 3.2 to show the error: https://ember-twiddle.com/6fea6c57693c2cd14312bdf4293f3778?openFiles=tests.integration.components.my-component-test.js%2Ctemplates.components.my-component.hbs

I'm using ember-fetch to get some data. If you change de depencies to ember version 3.1 the error do not occours.

[viniciussbs]

You people still have this issue? @Turbo87 @rwjblue the Ember Twiddle posted by @raphaelns-developer reproduces the issue.

[Mciocca]

Also running into the same issue. Also using ember-fetch and Ember 3.3

Test looks like the following.

test('after clicking on a request type', async function(assert){
    await visit('/report');
    await click('a');

    await settled();

    assert.equal(currentURL(), '/photo');
  });

[viniciussbs]

@raphaelns-developer works with me, he can correct me if I'm wrong, but this is how we solved the issue.

The problem was that ember-cli-update didn't updated the version of ember-test-helpers. Deleting package-lock.json and installing all the packages again did the trick. I guess he had another issue that was solved updating ember-test-helpers to the last release.

[Duder-onomy]

Just ran into this exact same prob.
I deleted the entry in my yarn.lock for @ember/test-helpers and re-ran yarn.
It pulled in 0.7.25 and everything works now.

[ggayowsky]

I will add my two cents here, however, I believe this is more of a problem with settled() than it is click.

I have a link-to component which transitions routes, and the click (settled) is returning control to the test between model and afterModel.

I tried to create an ember-twiddle to show my problem, however, twiddle is not properly loading the visit function from @ember/test-helpers

templates/my-route.hbs

{{#link-to "failing-route" class="failing-route-link"}}

routes/failing-route/view.js

model() {
  const myModel = {};
  return new EmberPromise((resolve) => {
    setTimeout(() => { resolve(model); }, 1);
  });
}

afterModel() {
  // Do stuff in after model
}

tests/acceptance/failing-route-test.js

test('transition to failing route', async function(assert) {
  await visit('/my-route');
  await click('.failing-route-link');
  assert.equal(currentRouteName(), 'failing-route.view', 'Should be at the failing route'); // This fails with currentRouteName return 'failing-route.loading'
});

If I change the model hook to return the model directly, or return Promise.resolve(model) the test passes. Otherwise, it fails.

[Turbo87]

@viniciussbs the linked twiddle does not seem to reproduce it for me 🤔

@ggayowsky in your snippet you're never resolving the returned promise

@all since this appears to be solved with the most recent releases I'll close this as solved. if you run into this issue, please make sure you don't have @ember/test-helpers locked to an older version.