Skip to content

Commit

Permalink
Merge pull request #55 from mk868/test-metadata
Browse files Browse the repository at this point in the history
Abstraction for reading test metadata added
  • Loading branch information
bitcoder authored Aug 20, 2024
2 parents e79bc91 + eeb1f60 commit c1eb83f
Show file tree
Hide file tree
Showing 10 changed files with 778 additions and 53 deletions.
75 changes: 74 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ If you want, you may configure certain aspects of this extension. The defaults s
- `report_directory`: the directory where to generate the report, in relative or absolute format. Default is "target"
- `add_timestamp_to_report_filename`: add a timestamp based suffix to the report. Default is "false".
- `report_only_annotated_tests`: only include tests annotated with @XrayTest or @Requirement. Default is "false".
- `reports_per_class`: generate JUnit XML reports per test class instead of a single report with all results; if true, `report_filename`, and `add_timestamp_to_report_filename` are ignored. Default is "false".
- `reports_per_class`: generate JUnit XML reports per test class instead of a single report with all results; if true, `report_filename`, and `add_timestamp_to_report_filename` are ignored. Default is "false".
- `test_metadata_reader`: override the default logic responsible for reading meta-information about test methods.

Example:

Expand All @@ -72,6 +73,7 @@ report_directory=reports
add_timestamp_to_report_filename=true
report_only_annotated_tests=false
reports_per_class=false
test_metadata_reader=com.example.CustomTestMetadataReader
```

## How to use
Expand Down Expand Up @@ -190,6 +192,74 @@ public class XrayEnabledTestExamples {
}
```

### Customizing how test metadata is read

When generating the report, it's allowed to customize the way the test method information is read.
By default, test information such as id, key, summary, description and requirements are read directly from @XrayTest and @Requirements annotations.
This behavior can be overridden by the user when he wants to change the way these meta-information are generated, or when he wants to use their own annotations to describe the tests.
To do this, you need to create a public class with a no-argument constructor that implements the `app.getxray.xray.junit.customjunitxml.XrayTestMetadataReader` interface (or extend `app.getxray.xray.junit.customjunitxml.DefaultXrayTestMetadataReader` class).
Then must add `test_metadata_reader` entry with the class name to the `xray-junit-extensions.properties` file.
#### Example: Custom test metadata reader to read Jira key from custom @JiraKey annotation
_JiraKey.java_
```java
package com.example;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
@Retention(RetentionPolicy.RUNTIME)
public @interface JiraKey {
String value();
}
```
_CustomTestMetadataReader.java_
```java
package com.example;
import app.getxray.xray.junit.customjunitxml.DefaultXrayTestMetadataReader;
import org.junit.platform.launcher.TestIdentifier;
import java.util.Optional;
public class CustomTestMetadataReader extends DefaultXrayTestMetadataReader {
@Override
public Optional<String> getKey(TestIdentifier testIdentifier) {
return getTestMethodAnnotation(testIdentifier, JiraKey.class)
.map(JiraKey::value)
.filter(s -> !s.isEmpty());
}
}
```
_xray-junit-extensions.properties_
```
test_metadata_reader=com.example.CustomTestMetadataReader
```
_SimpleTest.java_
```java
package com.example;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
class SimpleTest {
@Test
@JiraKey("CALC-123")
@DisplayName("simple test")
void simpleTest() {
// ...
}
}
```
## Other features and limitations
### Name of Tests
Expand All @@ -200,6 +270,9 @@ The summary of the Test issue will be set based on these rules (the first that a
* based on the `@DisplayName` annotation, or the display name of dynamically created tests from a TestFactory;
* based on the test's method name.

> [!TIP]
> This behavior can be changed by defining a custom test metadata reader.

### Parameterized and repeated tests

For the time being, and similar to what happened with legacy JUnit XML reports produces with JUnit 4, parameterized tests (i.e. annotated with `@ParameterizedTest`) will be mapped to similar `<testcase>` elements in the JUnit XML report.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
/*
* Copyright 2024-2024 the original author or authors.
*
* All rights reserved. This program and the accompanying materials are
* made available under the terms of the Eclipse Public License v2.0 which
* accompanies this distribution and is available at
*
* https://www.eclipse.org/legal/epl-v20.html
*/

package app.getxray.xray.junit.customjunitxml;

import app.getxray.xray.junit.customjunitxml.annotations.Requirement;
import app.getxray.xray.junit.customjunitxml.annotations.XrayTest;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DisplayNameGeneration;
import org.junit.jupiter.api.TestFactory;
import org.junit.platform.commons.support.AnnotationSupport;
import org.junit.platform.engine.support.descriptor.MethodSource;
import org.junit.platform.launcher.TestIdentifier;

import java.lang.annotation.Annotation;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Optional;

public class DefaultXrayTestMetadataReader implements XrayTestMetadataReader {

@Override
public Optional<String> getId(TestIdentifier testIdentifier) {
return getTestMethodAnnotation(testIdentifier, XrayTest.class)
.map(XrayTest::id)
.filter(s -> !s.isEmpty());
}

@Override
public Optional<String> getKey(TestIdentifier testIdentifier) {
return getTestMethodAnnotation(testIdentifier, XrayTest.class)
.map(XrayTest::key)
.filter(s -> !s.isEmpty());
}

@Override
public Optional<String> getSummary(TestIdentifier testIdentifier) {
Optional<String> testSummary = getTestMethodAnnotation(testIdentifier, XrayTest.class)
.map(XrayTest::summary)
.filter(s -> !s.isEmpty());
if (testSummary.isPresent()) {
return testSummary;
}

Optional<DisplayName> displayName = getTestMethodAnnotation(testIdentifier, DisplayName.class);
if (displayName.isPresent()) {
return Optional.of(displayName.get().value());
}


Optional<TestFactory> dynamicTest = getTestMethodAnnotation(testIdentifier, TestFactory.class);
Optional<DisplayNameGeneration> displayNameGenerator = getTestClassAnnotation(testIdentifier, DisplayNameGeneration.class);
if (dynamicTest.isPresent() || displayNameGenerator.isPresent()) {
return Optional.of(testIdentifier.getDisplayName());
}

return Optional.empty();
}

@Override
public Optional<String> getDescription(TestIdentifier testIdentifier) {
return getTestMethodAnnotation(testIdentifier, XrayTest.class)
.map(XrayTest::description)
.filter(s -> !s.isEmpty());
}

@Override
public List<String> getRequirements(TestIdentifier testIdentifier) {
return getTestMethodAnnotation(testIdentifier, Requirement.class)
.map(Requirement::value)
.map(arr -> Collections.unmodifiableList(Arrays.asList(arr)))
.orElse(Collections.emptyList());
}

protected <A extends Annotation> Optional<A> getTestMethodAnnotation(TestIdentifier testIdentifier, Class<A> aClass) {
return testIdentifier.getSource()
.filter(a -> a instanceof MethodSource)
.map(MethodSource.class::cast)
.map(MethodSource::getJavaMethod)
.flatMap(a -> AnnotationSupport.findAnnotation(a, aClass));
}

protected <A extends Annotation> Optional<A> getTestClassAnnotation(TestIdentifier testIdentifier, Class<A> aClass) {
return testIdentifier.getSource()
.filter(a -> a instanceof MethodSource)
.map(MethodSource.class::cast)
.map(MethodSource::getJavaClass)
.flatMap(a -> AnnotationSupport.findAnnotation(a, aClass));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@

import javax.xml.stream.XMLStreamException;

import static org.junit.jupiter.api.DynamicTest.stream;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
Expand Down Expand Up @@ -65,6 +62,7 @@ public class EnhancedLegacyXmlReportGeneratingListener implements TestExecutionL
boolean addTimestampToReportFilename = false;
boolean reportOnlyAnnotatedTests = false;
boolean reportsPerClass = false;
XrayTestMetadataReader testInfoReader = new DefaultXrayTestMetadataReader();

private XmlReportData reportData;

Expand Down Expand Up @@ -95,6 +93,9 @@ public EnhancedLegacyXmlReportGeneratingListener(Path reportsDir, Path propertie
this.addTimestampToReportFilename = "true".equals(properties.getProperty("add_timestamp_to_report_filename"));
this.reportOnlyAnnotatedTests = "true".equals(properties.getProperty("report_only_annotated_tests", "false"));
this.reportsPerClass = "true".equals(properties.getProperty("reports_per_class", "false"));
if (!properties.getProperty("test_metadata_reader").isEmpty()) {
this.testInfoReader = (XrayTestMetadataReader) Class.forName(properties.getProperty("test_metadata_reader")).getConstructor().newInstance();
}
} else {
if (reportsDir == null) {
this.reportsDir = FileSystems.getDefault().getPath(DEFAULT_REPORTS_DIR);
Expand Down Expand Up @@ -183,7 +184,7 @@ private void writeXmlReportSafely(TestIdentifier testIdentifier, String rootName
xmlFile = this.reportsDir.resolve(fileName);

try (Writer fileWriter = Files.newBufferedWriter(xmlFile)) {
new XmlReportWriter(this.reportData, this.reportOnlyAnnotatedTests).writeXmlReport(testIdentifier, fileWriter);
new XmlReportWriter(this.reportData, this.reportOnlyAnnotatedTests, this.testInfoReader).writeXmlReport(testIdentifier, fileWriter);
} catch (XMLStreamException | IOException e) {
printException("Could not write XML report: " + xmlFile, e);
logger.error(e, () -> "Could not write XML report: " + xmlFile);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@

package app.getxray.xray.junit.customjunitxml;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DisplayNameGeneration;
import org.junit.jupiter.api.TestFactory;
import org.junit.platform.commons.logging.Logger;
import org.junit.platform.commons.logging.LoggerFactory;
import org.junit.platform.commons.support.AnnotationSupport;
Expand Down Expand Up @@ -96,14 +93,19 @@ class XmlReportWriter {
// re-add separator characters.
private static final Pattern CDATA_SPLIT_PATTERN = Pattern.compile("(?<=]])(?=>)");

private final XmlReportData reportData;
private static final Logger logger = LoggerFactory.getLogger(EnhancedLegacyXmlReportGeneratingListener.class);
private boolean reportOnlyAnnotatedTests = false;
private static final Logger logger = LoggerFactory.getLogger(EnhancedLegacyXmlReportGeneratingListener.class);

XmlReportWriter(XmlReportData reportData, boolean reportOnlyAnnotatedTests) {
this.reportData = reportData;
private final XmlReportData reportData;
private final boolean reportOnlyAnnotatedTests;
private final XrayTestMetadataReader xrayTestMetadataReader;

XmlReportWriter(XmlReportData reportData,
boolean reportOnlyAnnotatedTests,
XrayTestMetadataReader xrayTestMetadataReader) {
this.reportData = reportData;
this.reportOnlyAnnotatedTests = reportOnlyAnnotatedTests;
}
this.xrayTestMetadataReader = xrayTestMetadataReader;
}

void writeXmlReport(TestIdentifier rootDescriptor, Writer out) throws XMLStreamException {
TestPlan testPlan = this.reportData.getTestPlan();
Expand Down Expand Up @@ -234,7 +236,6 @@ private void writeTestcase(TestIdentifier testIdentifier, AggregatedTestResult t

final Optional<TestSource> testSource = testIdentifier.getSource();
final Optional<Method> testMethod = testSource.flatMap(this::getTestMethod);
final Class testClass = ((MethodSource)testSource.get()).getJavaClass();
Optional<XrayTest> xrayTest = AnnotationSupport.findAnnotation(testMethod, XrayTest.class);
Optional<Requirement> requirement = AnnotationSupport.findAnnotation(testMethod, Requirement.class);
if (reportOnlyAnnotatedTests && (!requirement.isPresent() && !xrayTest.isPresent())) {
Expand Down Expand Up @@ -272,49 +273,28 @@ private void writeTestcase(TestIdentifier testIdentifier, AggregatedTestResult t
newLine(writer);
}

// final Optional<Class<?>> testClass = testSource.flatMap(this::getTestClass);

if (requirement.isPresent()) {
String[] requirements = requirement.get().value();
List<String> requirements = xrayTestMetadataReader.getRequirements(testIdentifier);
if (!requirements.isEmpty()) {
addProperty(writer, "requirements", String.join(",", requirements));
}

String test_key = null;
String test_id = null;
String test_summary = null;
String test_description = null;
if (xrayTest.isPresent()) {
test_key = xrayTest.get().key();
if ((test_key != null) && (!test_key.isEmpty())) {
addProperty(writer, "test_key", test_key);
}

test_id = xrayTest.get().id();
if ((test_id != null) && (!test_id.isEmpty())) {
addProperty(writer, "test_id", test_id);
}

test_summary = xrayTest.get().summary();
test_description = xrayTest.get().description();
if ((test_description != null) && (!test_description.isEmpty())) {
addPropertyWithInnerContent(writer, "test_description", test_description);
}
}

Optional<TestFactory> dynamicTest = AnnotationSupport.findAnnotation(testMethod, TestFactory.class);
Optional<DisplayName> displayName = AnnotationSupport.findAnnotation(testMethod, DisplayName.class);
Optional<DisplayNameGeneration> displayNameGenerator = AnnotationSupport.findAnnotation(testClass, DisplayNameGeneration.class);
Optional<String> testKeyOpt = xrayTestMetadataReader.getKey(testIdentifier);
if (testKeyOpt.isPresent()) {
addProperty(writer, "test_key", testKeyOpt.get());
}
Optional<String> testIdOpt = xrayTestMetadataReader.getId(testIdentifier);
if (testIdOpt.isPresent()) {
addProperty(writer, "test_id", testIdOpt.get());
}

// this logic should be improved/simplified; the displayNameGererator logic isnt handling all cases, inclusing if it was set globally
if ( ((test_summary == null) || (test_summary.isEmpty())) && (displayName.isPresent()) ) {
test_summary = displayName.get().value();
}
if ( ((test_summary == null) || (test_summary.isEmpty())) && (dynamicTest.isPresent() || displayNameGenerator.isPresent()) ) {
test_summary = testIdentifier.getDisplayName();
}
Optional<String> testDescriptionOpt = xrayTestMetadataReader.getDescription(testIdentifier);
if (testDescriptionOpt.isPresent()) {
addPropertyWithInnerContent(writer, "test_description", testDescriptionOpt.get());
}

if ((test_summary != null) && (!test_summary.isEmpty())) {
addProperty(writer, "test_summary", test_summary);
Optional<String> testSummaryOpt = xrayTestMetadataReader.getSummary(testIdentifier);
if (testSummaryOpt.isPresent()) {
addProperty(writer, "test_summary", testSummaryOpt.get());
}

List<String> tags = testIdentifier.getTags().stream().map(TestTag::getName).map(String::trim)
Expand Down
Loading

0 comments on commit c1eb83f

Please sign in to comment.