Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open test reporting XML format enabled doesn't work #4313

Closed
gtach2o opened this issue Feb 12, 2025 · 13 comments
Closed

Open test reporting XML format enabled doesn't work #4313

gtach2o opened this issue Feb 12, 2025 · 13 comments

Comments

@gtach2o
Copy link

gtach2o commented Feb 12, 2025

That's by design. For compatibility with existing tools the legacy XML format outputs the legacy reporting name.

Please give the new Open Test Reporting XML format a try to get support for display names, report entries, hierarchical test structures, etc.

I added --config=junit.platform.reporting.open.xml.enabled=true, but seems like it doesn't work

WARNING: TestExecutionListener [org.junit.platform.reporting.open.xml.OpenTestReportGeneratingListener] threw exception for method: executionFinished(TestIdentifier [uniqueId = [engine:junit-jupiter], parentId = null, displayName = 'JUnit Jupiter', legacyReportingName = 'JUnit Jupiter', source = null, tags = [], type = CONTAINER], TestExecutionResult [status = SUCCESSFUL, throwable = null])
java.lang.RuntimeException: Failed to write event: Element{domElement=[e:finished: null]}
        at org.junit.platform.reporting.shadow.org.opentest4j.reporting.events.api.DefaultDocumentWriter.append(DefaultDocumentWriter.java:118)
        at org.junit.platform.reporting.shadow.org.opentest4j.reporting.events.api.DefaultDocumentWriter.append(DefaultDocumentWriter.java:42)
        at org.junit.platform.reporting.open.xml.OpenTestReportGeneratingListener.executionFinished(OpenTestReportGeneratingListener.java:247)
        at org.junit.platform.launcher.core.CompositeTestExecutionListener.lambda$executionFinished$10(CompositeTestExecutionListener.java:73)
        at org.junit.platform.launcher.core.CompositeTestExecutionListener.lambda$notifyEach$19(CompositeTestExecutionListener.java:102)
        at org.junit.platform.commons.util.CollectionUtils.forEachInReverseOrder(CollectionUtils.java:244)
        at org.junit.platform.launcher.core.IterationOrder$2.forEach(IterationOrder.java:30)
        at org.junit.platform.launcher.core.CompositeTestExecutionListener.notifyEach(CompositeTestExecutionListener.java:100)
        at org.junit.platform.launcher.core.CompositeTestExecutionListener.executionFinished(CompositeTestExecutionListener.java:72)
        at org.junit.platform.launcher.core.ExecutionListenerAdapter.executionFinished(ExecutionListenerAdapter.java:56)
        at org.junit.platform.launcher.core.CompositeEngineExecutionListener.lambda$executionFinished$6(CompositeEngineExecutionListener.java:59)
        at org.junit.platform.launcher.core.CompositeEngineExecutionListener.lambda$notifyEach$11(CompositeEngineExecutionListener.java:74)
        at org.junit.platform.commons.util.CollectionUtils.forEachInReverseOrder(CollectionUtils.java:244)
        at org.junit.platform.launcher.core.IterationOrder$2.forEach(IterationOrder.java:30)
        at org.junit.platform.launcher.core.CompositeEngineExecutionListener.notifyEach(CompositeEngineExecutionListener.java:72)
        at org.junit.platform.launcher.core.CompositeEngineExecutionListener.executionFinished(CompositeEngineExecutionListener.java:58)
        at org.junit.platform.launcher.core.DelegatingEngineExecutionListener.executionFinished(DelegatingEngineExecutionListener.java:46)
        at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.executionFinished(StackTracePruningEngineExecutionListener.java:46)
        at org.junit.platform.launcher.core.DelegatingEngineExecutionListener.executionFinished(DelegatingEngineExecutionListener.java:46)
        at org.junit.platform.launcher.core.OutcomeDelayingEngineExecutionListener.reportEngineOutcome(OutcomeDelayingEngineExecutionListener.java:69)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:199)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
        at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
        at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
        at org.junit.platform.console.tasks.ConsoleTestExecutor.executeTests(ConsoleTestExecutor.java:102)
        at org.junit.platform.console.tasks.ConsoleTestExecutor.lambda$execute$1(ConsoleTestExecutor.java:71)
        at org.junit.platform.console.tasks.CustomContextClassLoaderExecutor.replaceThreadContextClassLoaderAndInvoke(CustomContextClassLoaderExecutor.java:42)
        at org.junit.platform.console.tasks.CustomContextClassLoaderExecutor.invoke(CustomContextClassLoaderExecutor.java:33)
        at org.junit.platform.console.tasks.ConsoleTestExecutor.execute(ConsoleTestExecutor.java:71)
        at org.junit.platform.console.options.ExecuteTestsCommand.execute(ExecuteTestsCommand.java:61)
        at org.junit.platform.console.options.ExecuteTestsCommand.execute(ExecuteTestsCommand.java:27)
        at org.junit.platform.console.options.BaseCommand.call(BaseCommand.java:81)
        at org.junit.platform.console.shadow.picocli.CommandLine.executeUserObject(CommandLine.java:2045)
        at org.junit.platform.console.shadow.picocli.CommandLine.access$1500(CommandLine.java:148)
        at org.junit.platform.console.shadow.picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2465)
        at org.junit.platform.console.shadow.picocli.CommandLine$RunLast.handle(CommandLine.java:2457)
        at org.junit.platform.console.shadow.picocli.CommandLine$RunLast.handle(CommandLine.java:2419)
        at org.junit.platform.console.shadow.picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
        at org.junit.platform.console.shadow.picocli.CommandLine$RunLast.execute(CommandLine.java:2421)
        at org.junit.platform.console.shadow.picocli.CommandLine.execute(CommandLine.java:2174)
        at org.junit.platform.console.options.MainCommand.runCommand(MainCommand.java:146)
        at org.junit.platform.console.options.MainCommand.run(MainCommand.java:139)
        at org.junit.platform.console.options.CommandFacade.run(CommandFacade.java:54)
        at org.junit.platform.console.options.CommandFacade.run(CommandFacade.java:37)
        at org.junit.platform.console.ConsoleLauncher.main(ConsoleLauncher.java:33)
Caused by: java.lang.NullPointerException: Cannot invoke "java.io.Writer.flush()" because "this.fWriter" is null
        at java.xml/com.sun.xml.internal.stream.writers.XMLStreamWriterImpl.flush(XMLStreamWriterImpl.java:397)
        at org.junit.platform.reporting.shadow.org.opentest4j.reporting.events.api.DefaultDocumentWriter.writeNewLine(DefaultDocumentWriter.java:139)
        at org.junit.platform.reporting.shadow.org.opentest4j.reporting.events.api.DefaultDocumentWriter.append(DefaultDocumentWriter.java:112)
        ... 50 more

junit-platform-console-standalone-1.11.4.jar

Originally posted by @gtach2o in #4312

@marcphilipp
Copy link
Member

I'm unable to reproduce this. Could you please share a full example? Did a partial junit-platform-events-*.xml file get written?

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

I'm unable to reproduce this. Could you please share a full example? Did a partial junit-platform-events-*.xml file get written?

yes

<?xml version="1.0" ?>
<e:events xmlns="https://schemas.opentest4j.org/reporting/core/0.1.0"
          xmlns:e="https://schemas.opentest4j.org/reporting/events/0.1.0"
          xmlns:java="https://schemas.opentest4j.org/reporting/java/0.1.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
  <infrastructure>
    <hostName>foo</hostName>
    <userName>bar</userName>
    <operatingSystem>Mac OS X</operatingSystem>
    <cpuCores>10</cpuCores>
    <java:javaVersion>23.0.2</java:javaVersion>
    <java:fileEncoding>UTF-8</java:fileEncoding>
    <java:heapSize max="8589934592"></java:heapSize>
  </infrastructure>

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

test method example

@TestFactory
    Stream<DynamicTest> generateDynamicTests() {
        List<String> values = List.of("Test A", "Test B", "Test C");

        return IntStream.range(0, values.size()).mapToObj(index -> {
            String testName = "Test-" + index + " [" + values.get(index) + "]";
            return dynamicTest(testName, () -> {
                assertTrue(values.get(index).startsWith("Test"));
            });
        });
    }

run command

java -jar build/junit-platform-console-standalone-1.11.4.jar execute \
    --classpath build/libs/tests.jar \
    --select-class=foo.bar.TestNameClass --include-classname=.* \
    --disable-ansi-colors --details-theme=unicode --disable-banner \
    --config=junit.platform.reporting.open.xml.enabled=true \
    --single-color --details=flat --include-engine=junit-jupiter --reports-dir $TEST_ARTIFACTS

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

legacy xml report is fine

    <testcase name="generateDynamicTests()[1]" classname="foo.bar.TestNameClass" time="0.002">
        <system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:foo.bar.TestNameClass]/[test-factory:generateDynamicTests()]/[dynamic-test:#1]
display-name: Test-0 [Test A]
]]></system-out>
    </testcase>
    <testcase name="generateDynamicTests()[3]" classname="foo.bar.TestNameClass" time="0">
        <system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:foo.bar.TestNameClass]/[test-factory:generateDynamicTests()]/[dynamic-test:#3]
display-name: Test-2 [Test C]
]]></system-out>
    </testcase>
    <testcase name="generateDynamicTests()[2]" classname="foo.bar.TestNameClass" time="0">
        <system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:foo.bar.TestNameClass]/[test-factory:generateDynamicTests()]/[dynamic-test:#2]
display-name: Test-1 [Test B]
]]></system-out>
    </testcase>

@marcphilipp
Copy link
Member

That produces this for me:

<?xml version="1.0" ?>
<e:events xmlns="https://schemas.opentest4j.org/reporting/core/0.1.0" xmlns:e="https://schemas.opentest4j.org/reporting/events/0.1.0" xmlns:java="https://schemas.opentest4j.org/reporting/java/0.1.0" xmlns:junit="https://schemas.junit.org/open-test-reporting" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://schemas.junit.org/open-test-reporting https://junit.org/junit5/schemas/open-test-reporting/junit-1.9.xsd">
<infrastructure><hostName>...</hostName><userName>marc</userName><operatingSystem>Linux</operatingSystem><cpuCores>16</cpuCores><java:javaVersion>23.0.2</java:javaVersion><java:fileEncoding>UTF-8</java:fileEncoding><java:heapSize max="16642998272"></java:heapSize></infrastructure>
<e:started id="1" name="JUnit Jupiter" time="2025-02-12T11:34:39.710211313Z"><metadata><junit:uniqueId>[engine:junit-jupiter]</junit:uniqueId><junit:legacyReportingName>JUnit Jupiter</junit:legacyReportingName><junit:type>CONTAINER</junit:type></metadata></e:started>
<e:started id="2" name="JupiterTests" parentId="1" time="2025-02-12T11:34:39.723661620Z"><metadata><junit:uniqueId>[engine:junit-jupiter]/[class:junit.jupiter.JupiterTests]</junit:uniqueId><junit:legacyReportingName>junit.jupiter.JupiterTests</junit:legacyReportingName><junit:type>CONTAINER</junit:type></metadata><sources><java:classSource className="junit.jupiter.JupiterTests"></java:classSource></sources></e:started>
<e:started id="3" name="generateDynamicTests()" parentId="2" time="2025-02-12T11:34:39.731520789Z"><metadata><junit:uniqueId>[engine:junit-jupiter]/[class:junit.jupiter.JupiterTests]/[test-factory:generateDynamicTests()]</junit:uniqueId><junit:legacyReportingName>generateDynamicTests()</junit:legacyReportingName><junit:type>CONTAINER</junit:type></metadata><sources><java:methodSource className="junit.jupiter.JupiterTests" methodName="generateDynamicTests" methodParameterTypes=""></java:methodSource></sources></e:started>
<e:started id="4" name="Test-0 [Test A]" parentId="3" time="2025-02-12T11:34:39.751487520Z"><metadata><junit:uniqueId>[engine:junit-jupiter]/[class:junit.jupiter.JupiterTests]/[test-factory:generateDynamicTests()]/[dynamic-test:#1]</junit:uniqueId><junit:legacyReportingName>generateDynamicTests()[1]</junit:legacyReportingName><junit:type>TEST</junit:type></metadata><sources><java:methodSource className="junit.jupiter.JupiterTests" methodName="generateDynamicTests" methodParameterTypes=""></java:methodSource></sources></e:started>
<e:finished id="4" time="2025-02-12T11:34:39.757689540Z"><result status="SUCCESSFUL"></result></e:finished>
<e:started id="5" name="Test-1 [Test B]" parentId="3" time="2025-02-12T11:34:39.759501311Z"><metadata><junit:uniqueId>[engine:junit-jupiter]/[class:junit.jupiter.JupiterTests]/[test-factory:generateDynamicTests()]/[dynamic-test:#2]</junit:uniqueId><junit:legacyReportingName>generateDynamicTests()[2]</junit:legacyReportingName><junit:type>TEST</junit:type></metadata><sources><java:methodSource className="junit.jupiter.JupiterTests" methodName="generateDynamicTests" methodParameterTypes=""></java:methodSource></sources></e:started>
<e:finished id="5" time="2025-02-12T11:34:39.760203497Z"><result status="SUCCESSFUL"></result></e:finished>
<e:started id="6" name="Test-2 [Test C]" parentId="3" time="2025-02-12T11:34:39.760651282Z"><metadata><junit:uniqueId>[engine:junit-jupiter]/[class:junit.jupiter.JupiterTests]/[test-factory:generateDynamicTests()]/[dynamic-test:#3]</junit:uniqueId><junit:legacyReportingName>generateDynamicTests()[3]</junit:legacyReportingName><junit:type>TEST</junit:type></metadata><sources><java:methodSource className="junit.jupiter.JupiterTests" methodName="generateDynamicTests" methodParameterTypes=""></java:methodSource></sources></e:started>
<e:finished id="6" time="2025-02-12T11:34:39.761198556Z"><result status="SUCCESSFUL"></result></e:finished>
<e:finished id="3" time="2025-02-12T11:34:39.763590632Z"><result status="SUCCESSFUL"></result></e:finished>
<e:finished id="2" time="2025-02-12T11:34:39.765473357Z"><result status="SUCCESSFUL"></result></e:finished>
<e:finished id="1" time="2025-02-12T11:34:39.766198737Z"><result status="SUCCESSFUL"></result></e:finished>
</e:events>

Is build/libs/tests.jar the only thing on your classpath? Which exact Java version and distribution are you using?

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

openjdk 23.0.2 2025-01-21
tests.jar is a jar built by shadow plugin

@marcphilipp
Copy link
Member

Please share the full output of java --version

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

The same error is for this java version.
I don't think it is somehow related.

java --version
openjdk 23.0.1 2024-10-15
OpenJDK Runtime Environment Homebrew (build 23.0.1)
OpenJDK 64-Bit Server VM Homebrew (build 23.0.1, mixed mode, sharing)

@marcphilipp
Copy link
Member

Well, since I cannot reproduce it, I'm unable to find the root cause and fix it.

Could you please extract a minimal, reproducible example from your project and share it as a zip file or a GitHub repo?

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

Well, since I cannot reproduce it, I'm unable to find the root cause and fix it.

Could you please extract a minimal, reproducible example from your project and share it as a zip file or a GitHub repo?

Seems like found the cause.
Could you add
implementation("net.sf.saxon:Saxon-HE:12.5") in dependencies and run again ?

@marcphilipp
Copy link
Member

marcphilipp commented Feb 12, 2025

@marcphilipp
Copy link
Member

@gtach2o I've fixed the root cause in open-test-reporting and have update the dependency in this repo to use a snapshot. Could you please try out junit-platform-console-standalone-1.13.0-20250212.152056-4.jar?

@gtach2o
Copy link
Author

gtach2o commented Feb 12, 2025

@gtach2o I've fixed the root cause in open-test-reporting and have update the dependency in this repo to use a snapshot. Could you please try out junit-platform-console-standalone-1.13.0-20250212.152056-4.jar?

It works now with no issue.
Thanks!

@marcphilipp marcphilipp added this to the 5.12.0 milestone Feb 12, 2025
marcphilipp added a commit that referenced this issue Feb 12, 2025
marcphilipp added a commit that referenced this issue Feb 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants