Static @BeforeEach method causes entire test plan to abort #971
Comments
|
Thanks for the bug report! I have confirmed that this is in fact a bug: a configuration error like that should not abort the entire test plan. |
|
As an aside, I had to repair your dependency configuration in your example project. The following works for me: dependencies {
compile 'org.junit.platform:junit-platform-engine:1.0.0-M6'
compile 'org.junit.platform:junit-platform-launcher:1.0.0-M6'
compile 'org.junit.jupiter:junit-jupiter-api:5.0.0-M6'
compile 'org.junit.jupiter:junit-jupiter-engine:5.0.0-M6'
} |
sbrannen
changed the title
|
Added Deliverables section |
5 tasks
sbrannen
changed the title
|
in progress |
sbrannen
added a commit
that referenced
this issue
Jul 21, 2017
Prior to this commit, configuration errors for lifecycle methods would result in the entire test plan being aborted during the discovery phase. Consequently, no tests would be executed in such scenarios. This commit addresses this issue by ensuring that configuration errors regarding lifecycle methods annotated with @BeforeAll, @afterall, BeforeEach, or @AfterEach are now reported during the execution of the affected test class. Issue: #971
|
Resolved in commit 6229d4c. |
ghost
removed
the
Andrei94
added a commit
to Andrei94/junit5
that referenced
this issue
Jul 24, 2017
commit 506e124 Author: Sam Brannen <[email protected]> Date: Mon Jul 24 16:31:58 2017 +0200 Add missing precondition check to ReflectionUtils.findMethod() commit 61d1af6 Author: Sam Brannen <[email protected]> Date: Mon Jul 24 15:56:34 2017 +0200 Introduce @disabled failing tests for overloaded default methods This commit introduces failing tests for finding methods in a class where a local method overrides a default method from an implemented interface. Issues: junit-team#978 commit fae5b6a Author: Sam Brannen <[email protected]> Date: Sun Jul 23 21:42:22 2017 +0200 Treat default methods like locally declared methods Prior to this commit, a non-overridden interface default method would incorrectly be considered overridden by a locally declared method that overloads the signature for the default method. This commit addresses this issue by treating default methods as if they were locally declared methods. In addition, this commit improves the documentation for ReflectionSupport and overhauls the internals of ReflectionUtils in the following ways. - removal of dead code - removal of superfluous precondition checks - consistent error messages for violated preconditions - enforcement of immutable return types where applicable - improved diagnostics for failed parameter type resolution - stable sorting of methods declared in interfaces Issue: junit-team#976 commit 90b0043 Author: Sam Brannen <[email protected]> Date: Sun Jul 23 17:15:12 2017 +0200 Simplify search algorithm in ReflectionUtils.findMethod() This commit modifies the search algorithm by reducing the search to a single iterative pass matching based on "compatible" signatures which already includes a preemptive check for exact matches anyway. Issues: junit-team#734, junit-team#969 commit 10f18d9 Author: Sam Brannen <[email protected]> Date: Sun Jul 23 13:33:34 2017 +0200 Improve search algorithm in ReflectionUtils.findMethod() Prior to this commit, it was impossible to select a method by "fully qualified method name" (FQMN) if the method was a generic default method from an interface implemented by a parameterized test class. It was also impossible to select such a method by class, method name, and parameter types. Thus, even though such methods would have been executed when executing the entire test class, such a method could not be executed alone (e.g., in an IDE). This commit addresses this issue by completely rewriting the implementation of findMethod() in ReflectionUtils. The algorithm is now iterative instead of recursive and stops searching once the first match is found. Furthermore, the algorithm actually makes two passes: - first pass: searches for a method whose signature matches exactly. - second pass: searches for a method whose signature is "compatible" with regard to sub-signatures and generics. The second pass is intended to find interface default methods that would otherwise not have been considered in the previous implementation of the search algorithm. As a positive side effect, this commit also improves the performance of look-ups for single methods within a test class hierarchy due to the new find-first search algorithm. Issues: junit-team#734, junit-team#969 commit 7e1f75a Author: Sam Brannen <[email protected]> Date: Sun Jul 23 15:01:15 2017 +0200 Introduce @disabled failing tests for finding overloaded methods This commit introduces failing tests for finding methods in a class where a local method overloads (but does not override) a default method from an implemented interface. Issue: junit-team#976 commit 20e0bb6 Author: Sam Brannen <[email protected]> Date: Sun Jul 23 14:29:24 2017 +0200 Add overloaded methods to BridgeMethodTests Issue: junit-team#976 commit 6229d4c Author: Sam Brannen <[email protected]> Date: Fri Jul 21 22:22:08 2017 +0200 Report lifecycle configuration errors during test execution Prior to this commit, configuration errors for lifecycle methods would result in the entire test plan being aborted during the discovery phase. Consequently, no tests would be executed in such scenarios. This commit addresses this issue by ensuring that configuration errors regarding lifecycle methods annotated with @BeforeAll, @afterall, BeforeEach, or @AfterEach are now reported during the execution of the affected test class. Issue: junit-team#971 commit 901cff9 Author: Sam Brannen <[email protected]> Date: Fri Jul 21 17:05:49 2017 +0200 Add TODOs for junit-team#969 commit 84d0fd8 Author: Sam Brannen <[email protected]> Date: Fri Jul 21 16:29:45 2017 +0200 Introduce failing tests to reproduce bug in findMethod() This commit introduces @Diabled/failing tests that reproduce the bug related to finding or selecting a method method by its "fully qualified method name" (FQMN) if the method is a parameterized default method from an interface implemented by a class. Issue: junit-team#969 commit 6845526 Author: Christian Stein <[email protected]> Date: Thu Jul 20 23:41:56 2017 +0200 Add release notes for 5.0.0-RC1 commit b243753 Author: Christian Stein <[email protected]> Date: Thu Jul 20 15:00:07 2017 +0200 Remove deprecated execute method from Launcher Prior to this commit the `execute(LauncherDiscoveryRequest)` method was temporarily available to restore binary compatibility with external tools using it. Now the method is removed. External tools are now forced to recompile against the new SNAPSHOT version of the JUnit Platform. Closes junit-team#740 commit a70d77f Author: Sam Brannen <[email protected]> Date: Wed Jul 19 17:13:13 2017 +0200 Add missing @API declaration to JUnitPlatformProvider Issue: junit-team#856 commit 147e073 Author: Sam Brannen <[email protected]> Date: Wed Jul 19 17:07:44 2017 +0200 Delete @API declarations on non-public types in junit-platform-console Issue: junit-team#856 commit 135eb4d Author: Sam Brannen <[email protected]> Date: Wed Jul 19 17:03:55 2017 +0200 Delete @API declarations on non-public types in junit-platform-commons Issue: junit-team#856 commit 825cc35 Author: Sam Brannen <[email protected]> Date: Wed Jul 19 16:51:10 2017 +0200 Delete @API declarations on non-public types in junit-jupiter-engine Issue: junit-team#856 commit 122771a Author: Sam Brannen <[email protected]> Date: Wed Jul 19 14:01:50 2017 +0200 Add generic parameter tests for ReflectionUtils.findMethod This commit introduces a test in ReflectionUtilsTests to verify the status quo regarding "fully qualified method names" and parameterized type information in the parameter list. Issue: junit-team#956 commit aaa28e3 Author: Sam Brannen <[email protected]> Date: Wed Jul 19 13:57:08 2017 +0200 Add tests for FQMN containing generic type info This commit introduces tests for selecting methods by their "fully qualified method names" in conjunction with ParameterResolver support for such methods. The variant that includes parameterized type information currently fails (as expected) and has therefore been @disabled until further notice. Issue: junit-team#956 commit 67df030 Author: Sam Brannen <[email protected]> Date: Wed Jul 19 13:50:30 2017 +0200 Polishing commit 438d268 Author: Christian Stein <[email protected]> Date: Wed Jul 19 01:10:20 2017 +0200 Add test for resolving parameterized Map types Closes junit-team#965 commit 70af29f Author: Jonathan Bluett-Duncan <[email protected]> Date: Tue Jul 18 20:16:53 2017 +0100 Use the modern java.time API to get build times commit 0cc3524 Author: Marc Philipp <[email protected]> Date: Tue Jul 18 22:03:20 2017 +0200 Update README to reflect new phase commit a7634d9 Author: Marc Philipp <[email protected]> Date: Tue Jul 18 21:53:16 2017 +0200 Fix Javadoc warnings for missing references commit f2ce358 Author: Marc Philipp <[email protected]> Date: Tue Jul 18 21:52:49 2017 +0200 Back to snapshots for further development commit 3e6482a Author: Marc Philipp <[email protected]> Date: Tue Jul 18 21:20:38 2017 +0200 Release 5.0.0-M6 commit e2160e7 Author: Marc Philipp <[email protected]> Date: Tue Jul 18 21:24:30 2017 +0200 Fix Javadoc link commit eea15b1 Author: Marc Philipp <[email protected]> Date: Tue Jul 18 21:20:25 2017 +0200 Remove duplicate "M" commit 28407da Author: Marc Philipp <[email protected]> Date: Tue Jul 18 20:49:02 2017 +0200 Finishing touches on M6 Release Notes commit 7f7f2c3 Author: Sam Brannen <[email protected]> Date: Tue Jul 18 21:11:47 2017 +0200 Polish 5.0 M6 release notes commit 8d85570 Author: Christian Stein <[email protected]> Date: Tue Jul 18 18:50:36 2017 +0200 Add Java 9 compatibility section commit 9406e44 Author: Sam Brannen <[email protected]> Date: Tue Jul 18 18:22:22 2017 +0200 Clean up warnings commit 430f2f6 Author: Sam Brannen <[email protected]> Date: Mon Jul 17 15:57:40 2017 +0200 Introduce and enforce syntax for tags Prior to this commit, tags were trimmed (junit-team#942) but were otherwise allowed to contain any type of character, including whitespace or ISO control characters. However, this degree of flexibility can lead to subtle configuration errors and also makes it challenging to reason about a tag expression language as proposed in junit-team#927. This commit addresses these issues by introducing and enforcing a syntax for tags. Specifically, all tags must now conform to the following syntax rules. * A tag must not be null or blank. * A trimmed tag must not contain whitespace. * A trimmed tag must not contain ISO control characters. Consequently, the following changes make it impossible to create a TestTag or TagFilter based on a tag that is not syntactically valid. * The TagFilter.includeTags(), TagFilter.excludeTags(), and TestTag.create() factory methods now throw a PreconditionViolationException if a supplied tag is not syntactically valid. * @tag declarations containing invalid tag syntax will now be logged as a warning but effectively ignored. Furthermore, a new TestTag.isValid(String) method has been introduced for checking if a tag is syntactically valid. This allows test engines to preemptively validate the syntax of a tag provided by a user before attempting to register such an invalid tag. Issue: junit-team#956 commit c450484 Author: Stefan Birkner <[email protected]> Date: Tue Jul 18 07:56:22 2017 +0200 Update Gradle from 4.0 to 4.0.1 This bug-fix release fixes a possible deadlock in parallel test execution. The build-scan plugin has been update from 1.7.4 to 1.8, too. This update was necessary because the old plugin is not compatible anymore. commit 84e6362 Author: Christian Stein <[email protected]> Date: Mon Jul 17 21:20:55 2017 +0200 Update gradle-versions-plugin to 0.15.0 Fixes junit-team#958
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Overview
It is probable that I've encountered a bug related to execution of batch of tests.
If one of these test classes has a
staticmethod annotated with@BeforeEach, the whole batch is skipped.Although the case with
staticmethod annotated with@BeforeEachis an error, it must notruin the rest of test plan.
Steps to Reproduce
The details are here: https://github.com/SqAutoTestTeam/possiblejunit5bug.git
Current behavior
Expected behavior
Related Issues
Deliverables
@BeforeAll,@AfterAll,@BeforeEach,@AfterEach) do not cause the entire test plan to abort prematurely.The text was updated successfully, but these errors were encountered: