RunCukesTest prevents the execution of other tests #304

vitormcruz · 2012-04-21T22:09:01Z

Tried on IntelliJ. When I have a project with both features and common unit tests, the features get executed, but the other tests don't. If I comment @RunWith(Cucumber.class) from the RunCukes, other tests run as usual.

aslakhellesoy · 2012-04-22T01:21:16Z

Can you prove it in a way that doesn't involve random trial and error on our part?

vitormcruz · 2012-04-22T05:55:21Z

Yes, I cloned cucumber-jvm and reproduced the error in the java-helloworld example, here is the link: https://github.com/vitormcruz/cucumber-jvm/tree/master/examples/java-helloworld

I investigate the problem a little more and cucumber isn't actually preventing JUnit to execute other tests, instead it is (apparently) preventing other tests to report its completion. The error stops from happening if I remove or comment the following snippet of code from RunCukesTest:

@Cucumber.Options(format = {"pretty", "html:target/cucumber"})

This error only happens with test classes that are in a package different than the RunCukesTest. Test classes that are on the same package of the RunCukesTest executes normally.

This was as far as I could get, hope I could help. :)

aslakhellesoy · 2012-04-22T09:21:34Z

It prevents other tests from reporting completion where to? Eclipse? IntelliJ Idea? Maven? Ant? Other?

vitormcruz · 2012-04-22T15:25:08Z

I tested on IntelliJ and Maven. It happened on both.

aslakhellesoy · 2012-04-22T22:48:48Z

Thanks for the details. I have managed to reproduce it now.

aslakhellesoy · 2012-04-23T10:38:56Z

It appears that using a format like @Cucumber.Options(format = {"progress"}) causes a regular JUnit test to be stuck at yellow, while this does not: @Cucumber.Options(format = {"progress:progress.log"})

This is very odd. It seems that as long as Cucumber formatters write to file everything is fine, but as soon as they write to STDOUT it messes up the IDE's junit reporting.

I'm not sure if this is a JUnit bug, an IDE bug or a Cucumber bug...

vitormcruz · 2012-04-23T21:51:17Z

I tried to use JUnit directly, and the problem still happens. So we can say it is either a JUnit or a Cucumber bug. The outputs where:

With conf on:

JUnit version 4.10
Feature: Hello World
.
Scenario: Say hello ←[90m# cucumber\examples\java\hello
world\helloworld.feature:2←[0m
←[90mGiven ←[0m←[90mI have a hello app with "←[0m←[90m←[1mHowdy←[0m←[90m"←[0
m ←[90m# HelloStepdefs.I_have_a_hello_app_with(String)←[0m
.←[1A ←[32mGiven ←[0m←[32mI have a hello app with "←[0m←[32m←[1mHowdy←[0m←[32
m"←[0m ←[90m# HelloStepdefs.I_have_a_hello_app_with(String)←[0m
←[90mWhen ←[0m←[90mI ask it to say hi←[0m ←[90m# HelloStepd
efs.I_ask_it_to_say_hi()←[0m
.←[1A ←[32mWhen ←[0m←[32mI ask it to say hi←[0m ←[90m# Hello
Stepdefs.I_ask_it_to_say_hi()←[0m
←[90mThen ←[0m←[90mit should answer with "←[0m←[90m←[1mHowdy World←[0m←[90m"
←[0m ←[90m# HelloStepdefs.it_should_answer_with(String)←[0m
.←[1A ←[32mThen ←[0m←[32mit should answer with "←[0m←[32m←[1mHowdy World←[0m←
[32m"←[0m ←[90m# HelloStepdefs.it_should_answer_with(String)←[0m

With conf off:

JUnit version 4.10
....
.
Time: 0,102

OK (5 tests)

aslakhellesoy · 2012-04-23T22:00:55Z

I'm not sure I understand how this output proves that the regular JUnit test is not running. Please turn off ANSI colouring with @Cucumber.Options(monochrome = true)

vitormcruz · 2012-04-23T22:57:51Z

With conf on:

E:\Projetos\cucumber-jvm\examples\java-helloworld\target>java -classpath junit-4.10.jar;.\test-classes;C:\Users\Vitor.m2\repository\info\cukes\cucumber-core\1.0.3\cucumber-core-1.0.3.jar;C:\Users\Vitor.m2\repository\info\cukes\cucumber-junit\1.0.3\cucumber-junit-1.0.3.jar;C:\Users\Vitor.m2\repository\info\cukes\gherkin\2.9.3\gherkin-2.9.3.jar;C:\Users\Vitor.m2\repository\info\cukes\cucumber-java\1.0.3\cucumber-java-1.0.3.jar;E:\Projetos\cucumber-jvm\examples\java-helloworld\target\classes;C:\Users\Vitor.m2\repository\info\cukes\cucumber-html\0.2.1\cucumber-html-0.2.1.jar org.junit.runner.JUnitCore cucumber.examples.java.helloworld.RunCukesTest cucumber.examples.java.test.ExampleTest
JUnit version 4.10
Feature: Hello World
.
Scenario: Say hello # cucumber\examples\java\helloworld\helloworld.feature:2
. Given I have a hello app with "Howdy" # HelloStepdefs.I_have_a_hello_app_with(String)
. When I ask it to say hi # HelloStepdefs.I_ask_it_to_say_hi()
. Then it should answer with "Howdy World" # HelloStepdefs.it_should_answer_with(String)

E:\Projetos\cucumber-jvm\examples\java-helloworld\target>

With conf off:

E:\Projetos\cucumber-jvm\examples\java-helloworld\target>java -classpath junit-4.10.jar;.\test-classes;C:\Users\Vitor.m2\repository\info\cukes\cucumber-core\1.0.3\cucumber-core-1.0.3.jar;C:\Users\Vitor.m2\repository\info\cukes\cucumber-junit\1.0.3\cucumber-junit-1.0.3.jar;C:\Users\Vitor.m2\repository\info\cukes\gherkin\2.9.3\gherkin-2.9.3.jar;C:\Users\Vitor.m2\repository\info\cukes\cucumber-java\1.0.3\cucumber-java-1.0.3.jar;E:\Projetos\cucumber-jvm\examples\java-helloworld\target\classes;C:\Users\Vitor.m2\repository\info\cukes\cucumber-html\0.2.1\cucumber-html-0.2.1.jar org.junit.runner.JUnitCore cucumber.examples.java.helloworld.RunCukesTest cucumber.examples.java.test.ExampleTest
JUnit version 4.10
....
.
Time: 0,104

OK (5 tests)

E:\Projetos\cucumber-jvm\examples\java-helloworld\target>

"I'm not sure I understand how this output proves that the regular JUnit test is not running."

JUnitCore uses a TextListerner to print those last lines of the test (If you want I can paste the code here):

Time: 0,104

OK (5 tests)

It seams to me that when cucumber conf is on, it prevents this part of the code to be executed. The report completion problem seams related with this output problem. Since the output problem occurs on IDEA, Maven and JUnit, I can only assume that the reporting problem persists on JUnit execution... I don't know how can I prove that, perhaps by implementing a custom listener? Anyway, the problem could be with some RunListener implemented by cucumber?

aslakhellesoy · 2012-04-24T07:28:19Z

I think a custom RunListener would be helpful to get to the bottom of this. If you want to try that out, please merge your branch with cucumber/master first so that the helloworld example uses the latest code.

vitormcruz · 2012-05-07T11:16:26Z

I did more investigantion and was unable to determine the source of the problem... It is very odd what is happening....

tmertens · 2012-10-22T22:10:51Z

I have run across this (or a similar) problem as well. I am not sure from the comment history if my issue is the same or a new/different issue than what the original author experienced.

I am running cucumber-jvm tests from a main() class using JUnitCore.run(MyTest.class). (I have my reasons) After I run a cucumber test using the "pretty" formatter to display output on the console, all console output thereafter is lost to the abyss.

It seems that what is happening is when the "pretty" formatter is used, after the test run is complete, the formatter.close() method is called to close the output stream(s) for the formatter. This effectively closes the Standard.out stream so that no additional output is written to the console from ANY output call for the rest of the program execution. When the formatter is closed, it should not close the default System.out output stream.

This includes System.out and the log4j ConsoleAppender output, etc. Anything using System.out for output will not be printed to the console.

This does not seem to affect JUnit itself, as it only seems to affect tests run using @RunWith(Cucumber.class), and furthermore it only affects those tests which have the "pretty" formatter specified. For example, if only HTML/JSON output formats are specified, this bug will not occur.

For example:

public class App {
public static void main(String[] args) {
JUnitCore junitRunner = new JUnitCore();
Result run;
run = junitRunner.run(com.example.RunTest.class); //Output for this test is printed to the terminal
run = junitRunner.run(com.example.RunTest2.class); //Output for this test is NOT printed to the terminal
System.out.println("This is not output to the terminal.");
}
}

Mishail · 2012-11-23T04:27:31Z

I've made quick and dirty patch which work well for me.

tmertens · 2012-11-27T15:36:55Z

Great! Any idea when this patch will make it to a release in the Maven repository?

aslakhellesoy · 2012-11-27T15:55:25Z

@tmertens - I have received #430 from @Mishail
I don't do time estimates. It will land when it lands ;-)

tmertens · 2012-11-27T15:57:47Z

Understood, thanks again for the fix :) This will be VERY helpful for me!

hieuduy · 2013-06-14T07:41:10Z

Hello,

I also experienced in this bug

Scenario: Say hello ←[90m# cucumber\examples\java\hello
world\helloworld.feature:2←[0m
←[90mGiven ←[0m←[90mI have a hello app with "←[0m←[90m←[1mHowdy←[0m←[90m"←[0
m ←[90m# HelloStepdefs.I_have_a_hello_app_with(String)←[0m
.←[1A ←[32mGiven ←[0m←[32mI have a hello app with "←[0m←[32m←[1mHowdy←[0m←[32
m"←[0m ←[90m# HelloStepdefs.I_have_a_hello_app_with(String)←[0m
←[90mWhen ←[0m←[90mI ask it to say hi←[0m ←[90m# HelloStepd
efs.I_ask_it_to_say_hi()←[0m
.←[1A ←[32mWhen ←[0m←[32mI ask it to say hi←[0m ←[90m# Hello
Stepdefs.I_ask_it_to_say_hi()←[0m
←[90mThen ←[0m←[90mit should answer with "←[0m←[90m←[1mHowdy World←[0m←[90m"
←[0m ←[90m# HelloStepdefs.it_should_answer_with(String)←[0m
.←[1A ←[32mThen ←[0m←[32mit should answer with "←[0m←[32m←[1mHowdy World←[0m←
[32m"←[0m ←[90m# HelloStepdefs.it_should_answer_with(String)←[0m

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at

java.util.ArrayList.rangeCheck(ArrayList.java:604) at

java.util.ArrayList.remove(ArrayList.java:445) at

gherkin.formatter.PrettyFormatter.indentedLocation(PrettyFormatter.ja va:142)

at gherkin.formatter.PrettyFormatter.printStep(PrettyFormatter.java:255)

at gherkin.formatter.PrettyFormatter.result(PrettyFormatter.java:199) at

sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

java:57) at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

sorImpl.java:43)

Not sure for what happened here but when I tried removing this option "pretty" in Test class then everything is passed fine.

veenamohandas · 2016-07-24T09:03:05Z

+1

veenamohandas · 2016-07-24T09:03:49Z

@RunWith(Cucumber.class)
@CucumberOptions(
features = "src/test/feature/", tags = Config.tags, plugin = { "pretty",
        "html:target/cucumber" })

 @notnow
  Scenario: To verify the Get request response time.                  �[90m# RestTest.feature:24�[0m
    �[32mGiven �[0m�[32mI am calling GetUser request.�[0m                               �[90m# Stepdefenition.i_am_calling_GetUser_request()�[0m
    �[32mWhen �[0m�[32mI am calling request with url parameter "�[0m�[32m�[1m1�[0m�[32m"�[0m                  �[90m# Stepdefenition.i_am_calling_GetUser_request_with_parameter(String)�[0m
    �[32mThen �[0m�[32mI fire the get request.�[0m                                      �[90m# Stepdefenition.i_fire_the_request()�[0m
    �[32mThen �[0m�[32mI should be able to get the response in "�[0m�[32m�[1m3612�[0m�[32m" milliSeconds.�[0m �[90m# Stepdefenition.i_should_be_able_to_get_the_response_in_milliSeconds(String)�[0m

mathan91 · 2017-06-02T07:35:18Z

I get the following error when I run through CLI runner,

Main.main(new String[]{"-g", "com.sanity.step.definition","-t", "@" +getRequest[1], "C:/Users/IBM_ADMIN/Documents/Automation/Performance Testing/PerformanceTesting/MyFeature.feature"});

I get the following ouput,
�[32m.�[0m�[32m.�[0m�[31mF�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m�[36m-�[0m

Also it just stops and doesn't execute the steps after that. I'm using Cucumber Core 1.1.2

lock · 2018-10-25T02:13:07Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Mishail pushed a commit to Mishail/cucumber-jvm that referenced this issue Nov 23, 2012

refs cucumber#304

c4809b2

aslakhellesoy closed this as completed in 8b3079a Nov 28, 2012

lock bot locked as resolved and limited conversation to collaborators Oct 25, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RunCukesTest prevents the execution of other tests #304

RunCukesTest prevents the execution of other tests #304

vitormcruz commented Apr 21, 2012

aslakhellesoy commented Apr 22, 2012

vitormcruz commented Apr 22, 2012

aslakhellesoy commented Apr 22, 2012

vitormcruz commented Apr 22, 2012

aslakhellesoy commented Apr 22, 2012

aslakhellesoy commented Apr 23, 2012

vitormcruz commented Apr 23, 2012

aslakhellesoy commented Apr 23, 2012

vitormcruz commented Apr 23, 2012

aslakhellesoy commented Apr 24, 2012

vitormcruz commented May 7, 2012

tmertens commented Oct 22, 2012

Mishail commented Nov 23, 2012

tmertens commented Nov 27, 2012

aslakhellesoy commented Nov 27, 2012

tmertens commented Nov 27, 2012

hieuduy commented Jun 14, 2013

veenamohandas commented Jul 24, 2016

veenamohandas commented Jul 24, 2016 •

edited

Loading

mathan91 commented Jun 2, 2017

lock bot commented Oct 25, 2018

RunCukesTest prevents the execution of other tests #304

RunCukesTest prevents the execution of other tests #304

Comments

vitormcruz commented Apr 21, 2012

aslakhellesoy commented Apr 22, 2012

vitormcruz commented Apr 22, 2012

aslakhellesoy commented Apr 22, 2012

vitormcruz commented Apr 22, 2012

aslakhellesoy commented Apr 22, 2012

aslakhellesoy commented Apr 23, 2012

vitormcruz commented Apr 23, 2012

aslakhellesoy commented Apr 23, 2012

vitormcruz commented Apr 23, 2012

aslakhellesoy commented Apr 24, 2012

vitormcruz commented May 7, 2012

tmertens commented Oct 22, 2012

Mishail commented Nov 23, 2012

tmertens commented Nov 27, 2012

aslakhellesoy commented Nov 27, 2012

tmertens commented Nov 27, 2012

hieuduy commented Jun 14, 2013

veenamohandas commented Jul 24, 2016

veenamohandas commented Jul 24, 2016 • edited Loading

mathan91 commented Jun 2, 2017

lock bot commented Oct 25, 2018

veenamohandas commented Jul 24, 2016 •

edited

Loading