Support for dry-run to print undefined steps

[TitouanVanBelle]

It would be nice to be able to scan through all the scenarios to make sure all steps are defined.

Given the following scenario

Feature: Test
  Scenario: Test
    Given this step is undefined

When I set Cucumberish.instance().dryRun to true
And I run the tests
Then it should pass and print the following in the console

We found some undefined steps in your scenarios. Use the following snippets to fine them:

Given("this step is undefined") { (args, userInfo) in
    
}

[brentleyjones]

That's a wonderful idea! Contributions are welcome.

[brentleyjones]

@TitouanVanBelle Should undefined steps in the dry run be a test failure? I think it would be good to fail each scenario that has undefined steps. That way you can do a dry run, if there are no failures you can do the real run. On failures you can skip the long run, which can speed up people's CI processes.

[TitouanVanBelle]

Could be an idea yeah, but I don't really see the dryRun as a feature that you would use on a CI. I personally use it to implement new scenarios but as soon as the code hit the CI I would assume all the steps are already defined.

[brentleyjones]

On our team we have people writing the feature files before other people write the step definitions. We use tags to make sure scenarios with incomplete step definitions are not run. Sometimes they forget the tags and we have to wait until the whole run is done to see that a failure was just because of a missing step definition.

Ideally we would do a step definition check as part of a normal run and fail early there as well, but that's a bigger architectural change. I'm fine with just setting up stuff locally to run a normal run after a successful dry run.

[TitouanVanBelle]

Ah ok got it. Then yeah, it would be good to fail the run if there are undefined steps.

[brentleyjones]

Released as part of 1.3.0. 🎉