how to enable generateTestResultAttributes using Maven #1673

bitcoder · 2018-01-26T13:44:18Z

TestNG Version

Note: only the latest version is supported
6.13.1

Expected behavior

testresult attributes should be present in the XML report

Actual behavior

there are no testresult attributes present in the XML report

Is the issue reproductible on runner?

Test case sample

Please, share the test case (as small as possible) which shows the issue

How can I setup TestNG, using Maven, for setting up the generateTestResultAttributes flag to true?
I tried several ways but I was unable to set it up; i'm unsure if there is a bug or not.
In my pom.xml I added this kind of config:

    <build>
        <plugins>
....
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.12.4</version>
        <configuration>
          <properties>
            <property>
              <name>usedefaultlisteners</name>
              <value>flase</value> <!-- disabling default listeners is optional -->
            </property>
             <property>
              <name>generateTestResultAttributes</name>
              <value>true</value>
            </property>
            <property>
              <name>reporter</name>
              <value>org.testng.reporters.XMLReporter</value>
            </property>
          </properties>
        </configuration>
      </plugin>

krmahadevan · 2018-01-26T13:49:53Z

@bitcoder - Its not clear what exactly is the problem. Can you please help elaborate a bit more on what is this issue all about ?

bitcoder · 2018-01-26T14:41:58Z

Sure.
I need to add some attributes to the test results in the generated XML file, such as the "requirement attribute".

    <test-method status="PASS" signature="CanAddNumbers()[pri:0, instance:com.xpand.java.CalcTest@e320068]" name="CanAddNumbers" duration-ms="2" started-at="2018-01-26T13:43:18Z" requirement="CALC-1" finished-at="2018-01-26T13:43:18Z">

For this I built an annotation listener which extends ITestListener and where I add an attribute after the test is run:

 */
public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
    if(method.isTestMethod()) {
        if(method.getClass().isAnnotationPresent(XrayAnnotation.class)) {
            System.out.println("This gets invoked after every TestNG Test that has @XrayAnnotation Annotation...");
            testResult.setAttribute("requirement",requirement);
        }
        if( !testSuccess ) {
            testResult.setStatus(ITestResult.FAILURE);
        }
    }
}

Anyway, I need that these testResult attributes be saved in the generated XML file.
I was trying to hack the XMLReporter from TestNG but I found this should already be possible if I enabled the "generateTestResultAttributes" flag as described here:
http://testng.org/doc/documentation-main.html#logging-reporters

Anyway, I'm using maven and I was unable to succesffuly make this attribute happen as expected.
Thus I'm unsure if it is a problem of on how to enable this in Maven or else if there is a bug with the "generateTestResultAttributes", where it does not behave as expected.
I just needed a way to make these testResult level attributes to be present in the generated XML report.

Here is my sample project:
java-testng-calc2.zip

krmahadevan · 2018-01-26T17:53:24Z

@bitcoder - I think the documentation is a bit outdated. I don't find any way in the codebase to basically enable what you are looking for.

That said and done, here's a way in which you can still get this done.

Sub-class org.testng.reporters.XMLReporter. The sub-class would not have anything, but its needed so that we have a reference to a class that we can explicitly inject and also configure to work with a different location for generating the reports.
Create your own variant of org.testng.ITestNGListenerFactory and within this factory, you add the customization for generating the custom attributes etc.,
Wire in this listener (created in (2)) via @Listeners or via service loaders.

Here's a sample that shows all of this in action.

import org.testng.ITestNGListener;
import org.testng.ITestNGListenerFactory;
import org.testng.Reporter;
import org.testng.TestNG;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
import org.testng.internal.ClassHelper;
import org.testng.reporters.XMLReporter;

import java.io.File;

@Listeners({Testclass.MyListenerFactory.class, Testclass.MyReporter.class})
public class Testclass {
    @Test
    public void testMethod() {
        System.err.println("Hello World");
        Reporter.getCurrentTestResult().setAttribute("Krishnan", "Mahadevan");
    }

    public static class MyListenerFactory implements ITestNGListenerFactory, ITestNGListener {

        @Override
        public ITestNGListener createListener(Class<? extends ITestNGListener> listenerClass) {
            System.err.println("Printing " + listenerClass.getName());
            ITestNGListener instance =  ClassHelper.newInstance(listenerClass);
            if (instance instanceof XMLReporter) {
                String current = TestNG.getDefault().getOutputDirectory() + File.separator + "new" + File.separator;
                ((XMLReporter) instance).getConfig().setOutputDirectory(current);
                ((XMLReporter) instance).getConfig().setGenerateTestResultAttributes(true);
            }
            return instance;
        }
    }

    public static class MyReporter extends XMLReporter {
    }
}

The resultant xml would look like below

<?xml version="1.0" encoding="UTF-8"?>
<testng-results skipped="0" failed="0" ignored="0" total="1" passed="1">
  <reporter-output>
  </reporter-output>
  <suite name="Default Suite" duration-ms="32" started-at="2018-01-26T17:52:33Z" finished-at="2018-01-26T17:52:33Z">
    <groups>
    </groups>
    <test name="testbed" duration-ms="32" started-at="2018-01-26T17:52:33Z" finished-at="2018-01-26T17:52:33Z">
      <class name="com.rationaleemotions.github.beanshellproblem.Testclass">
        <test-method status="PASS" signature="testMethod()[pri:0, instance:com.rationaleemotions.github.beanshellproblem.Testclass@262b2c86]" name="testMethod" duration-ms="8" started-at="2018-01-26T17:52:33Z" finished-at="2018-01-26T17:52:33Z">
          <reporter-output>
          </reporter-output>
          <attributes>
            <attribute name="Krishnan">
              <![CDATA[Mahadevan]]>
            </attribute> <!-- Krishnan -->
          </attributes>
        </test-method> <!-- testMethod -->
      </class> <!-- com.rationaleemotions.github.beanshellproblem.Testclass -->
    </test> <!-- testbed -->
  </suite> <!-- Default Suite -->
</testng-results>

bitcoder · 2018-01-30T13:52:29Z

Thanks @krmahadevan :)
I'm trying out the code you provided although I'm facing an issue setting up maven in order to use this reporter.
I'm not sure if you can provide me some guidance here, I tried following the approach as suggested in http://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html but unfortunately it does not work. Any clues how to setup this in pom.xml so it can be used with Maven?
Thanks

krmahadevan · 2018-01-30T14:35:17Z

@bitcoder - Have you tried following the section "Using Custom Listeners and Reporters" in that link ? That basically tells you how to wire in your listeners ( This reporter is also a listener)

If not, then you can try going through my blog which talks about setting up TestNG listeners in general via various different mechanisms

Blog link : https://rationaleemotions.wordpress.com/2012/01/27/listen-to-what-i-have-to-say-about-testng-listeners/

krmahadevan · 2018-01-30T14:53:18Z

@bitcoder
With respect to the listener injection not working via the <listeners> tag as detailed in the surefire plugin documentation that you shared, please check if it is related to https://issues.apache.org/jira/browse/SUREFIRE-1450 which talks about a similar problem when using surefire plugin with JDK9. Apparently this problem seems to have been resolved with surefire plugin Version 2.21.0

Kanaduchi · 2018-02-01T17:01:48Z

@krmahadevan
Is it possible to specify without writing own listeners but just as parameter of surfire?

Example,
<property> <name>generateTestResultAttributes</name> <value>true</value> </property>

Sometimes it is more better to enable some functionality just by setting parameter but not writing listeners

krmahadevan · 2018-02-02T04:25:40Z

@Kanaduchi - I hear you. But no that's not possible as of today. I dont see any mechanisms in the codebase that would facilitate this.

Kanaduchi · 2018-02-02T04:41:10Z

@krmahadevan
I found a suggestion.
If we add this parameter to CommandLineArgs class then we can use it with surfire.

Am I right?

I can try to modify code and test my suggestion

missedone · 2018-02-02T06:01:34Z

you can pass system properties in surefire plugin if the listener/reporter supports read parameters from system properties.

krmahadevan · 2018-02-04T09:25:59Z

@Kanaduchi - Thank you so much for taking the time to try and fix this issue. It was your PR that forced me to look at the codebase and get to the bottom of this. It looks like I was completely wrong all the time. The codebase does have provisions to support this as of today and hence there's no additional code changes needed. My humble apologies for providing the wrong information initially.

Here's how it can be done (I have found that it can be done ONLY via the Maven surefire plugin configuration i.e., by passing via surefire properties. )

Surefire plugin configuration would look like below

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
        </suiteXmlFiles>
        <properties>
            <property>
                <name>reporter</name>
                <value>org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true</value>
            </property>
        </properties>
    </configuration>
</plugin>

The value for the property name reporter should follow the following format :

FullyQualifiedClassName:<propertyName1>=<propertyValue1>,<propertyName2>=<propertyValue2>

Here

FullyQualifiedClassName - Represents the class name along with the complete package name of any implementation for org.testng.IReporter
<propertyName1> - Represents any property that can be invoked. For e.g., if there's a method called setFooValue(), then the property name would be <fooValue>

name value pair should be separated by =

sample test class looks like below

import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class GithubIssue1673Sample {
    @Test(groups = "regression")
    public void testMethod1() {
        System.err.println("testMethod1() executed");
        ITestResult result = Reporter.getCurrentTestResult();
        result.setAttribute("name", "Krishnan");
    }
    @Test(groups = "regression")
    public void testMethod2() {
        System.err.println("testMethod2() executed");
        ITestResult result = Reporter.getCurrentTestResult();
        result.setAttribute("name", "Mahadevan");
    }

}

suite xml looks like below

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="1673_Suite" parallel="false" verbose="2">
    <test name="1673_Test" verbose="2">
        <groups>
            <run>
                <include name="regression"/>
            </run>
        </groups>
        <classes>
            <class name="com.rationaleemotions.issue1673.GithubIssue1673Sample"/>
        </classes>
    </test>
</suite>

Here's the xml that was generated using TestNG 6.13.1

<?xml version="1.0" encoding="UTF-8"?>
<testng-results skipped="0" failed="0" ignored="0" total="2" passed="2">
  <reporter-output>
  </reporter-output>
  <suite name="1673_Suite" duration-ms="26" started-at="2018-02-04T09:12:54Z" finished-at="2018-02-04T09:12:54Z">
    <groups>
      <group name="regression">
        <method signature="GithubIssue1673Sample.testMethod1()[pri:0, instance:com.rationaleemotions.issue1673.GithubIssue1673Sample@3796751b]" name="testMethod1" class="com.rationaleemotions.issue1673.GithubIssue1673Sample"/>
        <method signature="GithubIssue1673Sample.testMethod2()[pri:0, instance:com.rationaleemotions.issue1673.GithubIssue1673Sample@3796751b]" name="testMethod2" class="com.rationaleemotions.issue1673.GithubIssue1673Sample"/>
      </group> <!-- regression -->
    </groups>
    <test name="1673_Test" duration-ms="26" started-at="2018-02-04T09:12:54Z" finished-at="2018-02-04T09:12:54Z">
      <class name="com.rationaleemotions.issue1673.GithubIssue1673Sample">
        <test-method groups="regression" status="PASS" signature="testMethod1()[pri:0, instance:com.rationaleemotions.issue1673.GithubIssue1673Sample@3796751b]" name="testMethod1" duration-ms="5" started-at="2018-02-04T14:42:54Z" finished-at="2018-02-04T14:42:54Z">
          <reporter-output>
          </reporter-output>
          <attributes>
            <attribute name="name">
              <![CDATA[Krishnan]]>
            </attribute> <!-- name -->
          </attributes>
        </test-method> <!-- testMethod1 -->
        <test-method groups="regression" status="PASS" signature="testMethod2()[pri:0, instance:com.rationaleemotions.issue1673.GithubIssue1673Sample@3796751b]" name="testMethod2" duration-ms="1" started-at="2018-02-04T14:42:54Z" finished-at="2018-02-04T14:42:54Z">
          <reporter-output>
          </reporter-output>
          <attributes>
            <attribute name="name">
              <![CDATA[Mahadevan]]>
            </attribute> <!-- name -->
          </attributes>
        </test-method> <!-- testMethod2 -->
      </class> <!-- com.rationaleemotions.issue1673.GithubIssue1673Sample -->
    </test> <!-- 1673_Test -->
  </suite> <!-- 1673_Suite -->
</testng-results>

krmahadevan · 2018-02-04T09:30:32Z

@bitcoder - Please refer to my above comments here to learn how to get this done.

missedone · 2018-02-04T11:25:00Z

@krmahadevan , your comment is really a good addition for the doc: [1], [2]

and the snippet of surefire configuration is quite good example as well, I think it's better to revise the documentation with these.

juherr · 2018-02-04T20:32:50Z

It could be an improvement for the surefire documentation too. I'm sure @Tibor17 will love it too.

Tibor17 · 2018-02-04T21:19:44Z

@juherr
The issue is long for me. What you need to improve in Surefire?

juherr · 2018-02-04T21:59:29Z

@Tibor17 The IDEA is just to improve the surefire documentation of reporters with a sample.

Tibor17 · 2018-02-04T22:03:24Z

@juherr
Alright, no problem. Can you be more concrete about the page and what you want me to to update there?

krmahadevan · 2018-02-05T03:16:53Z

@Tibor17

Here's the content that needs to be added

Here's how it can be done (I have found that it can be done ONLY via the Maven surefire plugin configuration i.e., by passing via surefire properties. )

Surefire plugin configuration would look like below

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
        </suiteXmlFiles>
        <properties>
            <property>
                <name>reporter</name>
                <value>org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true</value>
            </property>
        </properties>
    </configuration>
</plugin>

The value for the property name reporter should follow the following format :

FullyQualifiedClassName:<propertyName1>=<propertyValue1>,<propertyName2>=<propertyValue2>

Here

FullyQualifiedClassName - Represents the class name along with the complete package name of any implementation for org.testng.IReporter
<propertyName1> - Represents any property that can be invoked. For e.g., if there's a method called setFooValue(), then the property name would be <fooValue>

name value pair should be separated by =

Can we add a new page under http://maven.apache.org/surefire/maven-surefire-plugin/examples/ with its title as *Customizing TestNG reporters via Properties"

krmahadevan · 2018-02-05T04:16:34Z

@missedone for [1] there's already documentation available.

Here's the contents

Similar to the -listener option, except that it allows the configuration of JavaBeans-style properties on the reporter instance.
Example: -reporter com.test.MyReporter:methodFilter=insert,enableFiltering=true
You can have as many occurrences of this option, one for each reporter that needs to be added.

for [2] I have raised a new PR testng-team/testng-team.github.io#23

bitcoder · 2018-02-05T15:13:56Z

thanks @krmahadevan and everyone here involved!

vivekthangathurai · 2018-05-14T06:31:19Z

I am using this configuration

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version> 2.20.1</version>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
        <suiteXmlFiles>
            <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
        </suiteXmlFiles>
        <properties>
            <property>
                <name>usedefaultlisteners</name>
                <value>false</value>
                <!-- disabling default listeners is optional -->
            </property>
            <property>
                <name>reporter</name>
                <value>org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true</value>
            </property>
        </properties>
    </configuration>
</plugin>

but still the attributes are not generated. what could be the issue?

krmahadevan · 2018-05-14T15:49:41Z

@vivekthangathurai - You seem to have disabled the default reports via usedefaultlisteners set to false.. Can you please try removing that and see if that helps ?

vivekthangathurai · 2018-05-14T18:40:47Z

@krmahadevan tried but same result, no attributes generated. using Oxygen eclipse EE java.

Tibor17 · 2018-05-14T20:09:56Z

Surefire maps the properties in https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-testng/src/main/java/org/apache/maven/surefire/testng/conf/TestNGMapConfigurator.java

…

On Mon, May 14, 2018 at 8:40 PM, Vivek Thangathurai < ***@***.***> wrote: @krmahadevan <https://github.com/krmahadevan> tried but same result, no attributes generated. using Oxygen eclipse EE java. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1673 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA_yR-8xZGMDHbUJ759LtdGUThSkE29dks5tyc-3gaJpZM4RuUUM> .

-- Cheers Tibor

ssoban · 2019-11-13T08:23:13Z

@krmahadevan I know issue is closed but i need help on this and I am working on XRAY and facing the same issue. I tried the above solution but nothing happens.no attributes generated .....................
..............................................................
my test.java file:

package CalcTest;


import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.AssertJUnit;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.AssertJUnit;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.AssertJUnit;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.AssertJUnit;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.AssertJUnit;
import org.testng.Assert;
import org.testng.annotations.Test;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Listeners;
import org.testng.Reporter;
import org.testng.reporters.XMLReporter;
import org.testng.ITestResult;
import Xray_Automation.Demo.*;
import Xray_Automation.Demo_annotations.Xray;
import Xray_Automation.Demo_annotations.XrayListener;
 
@Test
//@Listeners({XrayListener.class})
public class TMTest2 {
 
    @BeforeMethod
	@BeforeSuite
    public void setUp() throws Exception {
 
    }
 
    
	@AfterMethod
	@AfterSuite
    public void tearDown() throws Exception {
    }
 
 
    @DataProvider
    public Object[][] ValidDataProvider() {
        return new Object[][]{
            {  1, 2, 3 },
            {  2, 3, 4 },  // error or the data itself :)
            { -1, 1, 0 }
        };
    }
 
 
    @Test(dataProvider = "ValidDataProvider")
    @Xray(requirement = "CALC-1234", test = "CALC-1")
    public void CanAddNumbersFromGivenData(final int a, final int b, final int c)
    {
        AssertJUnit.assertEquals(Calculator.Add(a, b), c);
       // System.out.println(requirement);
    }
 
 
    @Test
    @Xray(requirement = "CALC-1234", test = "CALC-2", labels = "core addition")
    public void CanAddNumbers()
    {
        AssertJUnit.assertEquals(Calculator.Add(1, 1),2);
        AssertJUnit.assertEquals(Calculator.Add(-1, 1),0);
    }
 
 
    @Test
    @Xray(requirement = "CALC-1235", labels = "core")
    public void CanSubtract()
    {
        AssertJUnit.assertEquals(Calculator.Subtract(1, 1), 0);
        AssertJUnit.assertEquals(Calculator.Subtract(-1, -1), 0);
        AssertJUnit.assertEquals(Calculator.Subtract(100, 5), 95);
    }
 
 
    @Test
    @Xray(requirement = "CALC-1236")
    public void CanMultiplyX()
    {
        AssertJUnit.assertEquals(Calculator.Multiply(1, 1), 1);
        AssertJUnit.assertEquals(Calculator.Multiply(-1, -1), 1);
        AssertJUnit.assertEquals(Calculator.Multiply(100, 5), 500);
    }
 
 
    @Test
    @Xray(requirement = "CALC-1237")
    public void CanDivide()
    {
        AssertJUnit.assertEquals(Calculator.Divide(1, 1), 1);
        AssertJUnit.assertEquals(Calculator.Divide(-1, -1), 1);
        AssertJUnit.assertEquals(Calculator.Divide(100, 5), 20);
    }
 
 
    @Test
    @Xray(requirement = "CALC-1237")
    public void CanDoStuff()
    {
        Assert.assertNotEquals(true, true);
    }
 
}

....................................................................................................................................

package Xray_Automation.Demo_annotations;

import java.lang.reflect.Method;

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;

import Rest_Request.ImportTesngResults;

import static java.lang.System.out;

import java.io.IOException;

import static java.lang.System.err;
 
/**
 * The listener interface for receiving Xray events.
 * The Listener can be automatically invoked when TestNG tests are run by using ServiceLoader mechanism.
 * You can also add this listener to a TestNG Test class by adding
 * <code>@Listeners({com.xpand.java.XrayAnnotationListener.class})</code>
 * before the test class
 *
 * @see Xray
 */
public class XrayListener implements IInvokedMethodListener, ITestListener  {
	String requirement=null;
	String test =null;
	String labels=null;
     
    boolean testSuccess = true;
     
     
    /* (non-Javadoc)
     * @see org.testng.IInvokedMethodListener#beforeInvocation(org.testng.IInvokedMethod, org.testng.ITestResult)
     */
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
  
    	
    	
        if(method.isTestMethod() && annotationPresent(method, Xray.class) ) {
        	
        	testResult.setAttribute("requirement", method.getTestMethod().getConstructorOrMethod().getMethod().getAnnotation(Xray.class).requirement()); 
     testResult.setAttribute("test", method.getTestMethod().getConstructorOrMethod().getMethod().getAnnotation(Xray.class).test());
           testResult.setAttribute("labels", method.getTestMethod().getConstructorOrMethod().getMethod().getAnnotation(Xray.class).labels());
            
         
        }
    	
         
    }
 
     
    private boolean annotationPresent(IInvokedMethod method, Class clazz) {
        boolean retVal = method.getTestMethod().getConstructorOrMethod().getMethod().isAnnotationPresent(clazz) ? true : false;
        return retVal;
    }
 
     
    /* (non-Javadoc)
     * @see org.testng.IInvokedMethodListener#afterInvocation(org.testng.IInvokedMethod, org.testng.ITestResult)
     */
    public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
    	 
        if(method.isTestMethod()) {
            if( !testSuccess ) {
                testResult.setStatus(ITestResult.FAILURE);
            }
        }
        
      
        
    }
 
    public void onTestStart(ITestResult result) {
        // TODO Auto-generated method stub
         
    }
 
    public void onTestSuccess(ITestResult result) {
        // TODO Auto-generated method stub
         
    }
 
    public void onTestFailure(ITestResult result) {
        // TODO Auto-generated method stub
         
    }
 
    public void onTestSkipped(ITestResult result) {
        // TODO Auto-generated method stub
         
    }
 
    public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
        // TODO Auto-generated method stub
         
    }
 
    public void onStart(ITestContext context) {
    	
    	 for(ITestNGMethod m1 : context.getAllTestMethods()) {
             if(m1.getConstructorOrMethod().getMethod().isAnnotationPresent(Xray.class)) {
                 //capture metadata information.
            	 requirement=  m1.getConstructorOrMethod().getMethod().getAnnotation(Xray.class).requirement();
            	 test= m1.getConstructorOrMethod().getMethod().getAnnotation(Xray.class).test();
                 labels= m1.getConstructorOrMethod().getMethod().getAnnotation(Xray.class).labels();
             }
         }
         
         
    }
 
    public void onFinish(ITestContext context) {
        // TODO Auto-generated method stub
    	
    	  ImportTesngResults callrest= new ImportTesngResults();
          try {
  			callrest.createtestexec();
  		} catch (IOException e) {
  			// TODO Auto-generated catch block
  			e.printStackTrace();
  		}
         
    }
     
}

.........................................................................................................................................................................
pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
 
   <groupId>Xray_Automation</groupId>
  <artifactId>Demo</artifactId>
  <version>0.0.1-SNAPSHOT</version>
 
 
    <properties>
         
        <!--XRay Properties -->
  
        <!--IN PROFILE ~.m2/settings.xml-->
        <!--<xray.jiraURL></xray.jiraURL>
        <xray.resultsFormat>JUNIT</xray.resultsFormat>
        <xray.username>syed</xray.username>
        <xray.password>123qwe</xray.password>-->
  
        <xray.projectKey>TM</xray.projectKey>
        <!--
        <xray.testExecKey></xray.testExecKey>
        <xray.testPlanKey></xray.testPlanKey>
        <xray.testEnvironments></xray.testEnvironments>
        <xray.revision></xray.revision>
        -->
  
        <xray.surefire.location>${basedir}/target/surefire-reports</xray.surefire.location>
        <!--End Xray Properties -->
  
    </properties>
 
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <debug>true</debug>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
             
            <plugin>
 
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M3</version>
 
                <configuration>
                    <testFailureIgnore>true</testFailureIgnore>
                    <suiteXmlFiles>
                      <suiteXmlFile>testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                    <properties>
                        <property>
                            <name>reporter</name>
                             <value>org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true</value>
                        </property>
                    </properties>
 
                </configuration>
             </plugin>
 
 
 
 
        </plugins>
    </build>
 
    <dependencies>
        <dependency>
          <groupId>org.testng</groupId>
          <artifactId>testng</artifactId>
          <version>6.11</version>
          <scope>test</scope>
        </dependency>
       
        
 
       
        <!-- https://mvnrepository.com/artifact/com.googlecode.json-simple/json-simple -->
<dependency>
    <groupId>com.googlecode.json-simple</groupId>
    <artifactId>json-simple</artifactId>
    <version>1.1.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/com.mashape.unirest/unirest-java -->
<dependency>
    <groupId>com.mashape.unirest</groupId>
    <artifactId>unirest-java</artifactId>
    <version>1.4.9</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpasyncclient-osgi -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpasyncclient-osgi</artifactId>
    <version>4.1.4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.10</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.json/json -->
<dependency>
    <groupId>org.json</groupId>
    <artifactId>json</artifactId>
    <version>20190722</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpcore</artifactId>
    <version>4.4.12</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.6</version>
</dependency>




<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpmime -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpmime</artifactId>
    <version>4.5.10</version>
</dependency>
 
       <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-surefire-report-plugin -->
<dependency>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-report-plugin</artifactId>
    <version>3.0.0-M3</version>
</dependency>
       
    </dependencies>
 
    
 
 
    <reporting>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <version>3.0.0-M3</version>
            </plugin>
            <plugin>
                <groupId>CalcTest</groupId>
                <artifactId>xray-maven-plugin</artifactId>
                <version>1.0.0</version>
            </plugin>
        </plugins>
    </reporting>
  
 
</project>

..........................................................................
testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">


  <test name="Test">
    <classes>
      <!--  class name="CalcTest.CalcTest"/-->
   
      <class name="CalcTest.TMTest2"/>
     
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

..............................................................
output sample is coming like:::

<test-method name="CanAddNumbersFromGivenData" signature="CanAddNumbersFromGivenData(int, int, int)[pri:0, instance:CalcTest.TMTest2@56ac3a89]" status="PASS" duration-ms="2" urso="CALC-1" finished-at="2019-11-13T07:03:13Z" started-at="2019-11-13T07:03:13Z" data-provider="ValidDataProvider" requirement="xpto">
          <params>
            <param index="0">
              <value>
                <![CDATA[1]]>
              </value>
            </param>
            <param index="1">
              <value>
                <![CDATA[2]]>
              </value>
            </param>
            <param index="2">
              <value>
                <![CDATA[3]]>
              </value>
            </param>
          </params>
          <reporter-output>
          </reporter-output>

No attribute is showing

juherr · 2019-11-13T08:58:14Z

@ssoban Please, could you open a new issue with a full runnable project?

bitcoder · 2019-11-13T09:03:13Z

Hi @ssoban ,
here is a full working example. Hope this helps :)
java-testng-calc-final.zip
AS @juherr , if you still face problems, please create another issue so the discussion can proceed there.
Regards
Sergio

Tibor17 · 2019-11-13T16:14:23Z

@krmahadevan
@juherr
@bitcoder
This notation org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true leads to a structural configuration. It would be maybe better to let the user to write some XML or YML file passed to the TestNG via Maven resource file or Surefire configuration. This would be capable of more extensions in the XML/YML format than only one line of string in Surefire properties. WDYT?

ChathaKiran · 2020-03-11T19:16:37Z

All of the responses are talking of the class name as "org.testng.reporters.XMLReporter". But I'm using the "org.testng.Reporter" class for generating the testng-results.xml. I'ven't explicitly used XMLReporter in my code. Even then, should I add below line in POM under SureFire Plugin Configuration. @krmahadevan

<property>
    <name>reporter</name
    <value>org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true</value>
</property>

krmahadevan · 2020-03-12T04:14:53Z

@ChathaKiran - The above configuration will basically enable dumping of attributes embedded in test results in the xml report (This is a default report that TestNG automatically wires in).

org.testng.Reporter - is not a reporter. Its a utility. It doesn't generate any testng-results.xml. This xml is generated by XMLReporter which is what the above mentioned configuration is referring to.

vlad230 · 2022-09-13T14:08:25Z

Hi guys,

I'm working on an integration between TestNG + Xray Jenkins plugin + Jira Xray in order to report results.
I'm using the maven-surefire-plugin integrated with Allure Reports as well:

<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.2</version>
                <configuration>
                    <testFailureIgnore>true</testFailureIgnore>
                    <suiteXmlFiles>
                        <suiteXmlFile>src\test\resources\suites\${suiteXmlFile}</suiteXmlFile>
                    </suiteXmlFiles>
                    <argLine>
                        -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    </argLine>

                    <properties>
                        <!-- Xray Property Start-->
                        <property>
                            <name>reporter</name>
                            <value>
                                org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true
                            </value>
                        </property>
                        <!-- Xray Property End-->
                    </properties>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>org.aspectj</groupId>
                        <artifactId>aspectjweaver</artifactId>
                        <version>${aspectj.version}</version>
                    </dependency>
                </dependencies>
            </plugin>

I have managed to get it working (it creates test executions, summary in Jira=test method name and passes the correct test status to them) using the TestNG XML multipart option but I would also like to push the @test method description to Jira Xray which is present inside the <test-method> tag.
One thing that I have noticed is that my testng-results.xml does NOT include the <attributes> tag. Should this be present?

.. <test-method signature="testPassing()[pri:0, instance:ui.tests.FreeTextEditorTest@7f9ab969]" started-at="2022-09-13T12:09:37 UTC" name="testPassing" description="Dummy test pass" finished-at="2022-09-13T12:09:37 UTC" duration-ms="1" status="PASS"> <reporter-output> </reporter-output> ...

I assume this needs to be configured in the Test Fields JSON inside the Jenkins "Xray Results import task" with some sort of variable that reads the description from the testng-results.xml and passes it to the Summary or Description fields in Jira:

{ "fields": { "description" : "Dummy test pass", "priority" : { "id": "3" }, "labels": [ "Regression", "Automation" ] } }

Any ideas on how to do this?

bitcoder · 2022-09-13T14:44:40Z

Hi @vlad230 ,
are you using the xray-testng-extensions package? You'll need that.
With it you can customize the summary and the description fields of the underlying Test issue in Xray/Jira.
You may find a working example here.

vlad230 · 2022-09-14T07:21:57Z

Hi @bitcoder,

Thanks a lot for the fast response :)
I am using these dependencies:

        <dependency>
            <groupId>com.xpandit.xray</groupId>
            <artifactId>xray-testng-extensions</artifactId>
            <version>1.1.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.xpandit.xray</groupId>
            <artifactId>xray-maven-plugin</artifactId>
            <version>1.1.0</version>
            <scope>test</scope>
        </dependency>

But maybe I need to somehow enable the listeners to generate/see the extra information in the testng-results.xml?

I have used it like this:

    @Xray(requirement = "ID-33", test = "ID-36")
    @Test(description = "Dummy test pass")
    public void testPassing() {
    }

I have tried to use the one mentioned in your link but Maven cannot resolve it and I cannot find it on maven central:

        <dependency>
            <groupId>app.getxray</groupId>
            <artifactId>xray-testng-extensions</artifactId>
            <version>0.1.0</version>
            <scope>test</scope>
        </dependency>

Was it removed from the public repository? I only see xray-maven-plugin & xray-junit-extensions from this groupId.
It seems the tutorial uses a SNAPSHOT version of it.

Also, is there a way to use the description defined in the @test annotation from TestNG as Jira summary/description? I need it to also to show up in Allure Reports and it uses it from here (of course I could duplicate the description in both annotations but was looking for a more elegant way).

bitcoder · 2022-09-14T10:39:31Z

Ok, so first of all, please discard these old dependencies:

        <dependency>
            <groupId>com.xpandit.xray</groupId>
            <artifactId>xray-testng-extensions</artifactId>
            <version>1.1.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.xpandit.xray</groupId>
            <artifactId>xray-maven-plugin</artifactId>
            <version>1.1.0</version>
            <scope>test</scope>
        </dependency>

Use this one instead:

        <dependency>
            <groupId>app.getxray</groupId>
            <artifactId>xray-testng-extensions</artifactId>
            <version>0.1.0</version>
            <scope>test</scope>
        </dependency>

It was not published on Maven Central (I was pending an internal ok); I just published it.
Note: if you really need the Xray maven plugin, there's a new open-source one: https://github.com/Xray-App/xray-maven-plugin, that is more powerful than the one you had listed.

Please note that the annotations changed with the new xray-testng-extensions package, as detailed in the docs.
You should now use @XrayTest and/or @Requirement. Previously you had just an annotation.

Concerning your last question, now you'll have something like this:

    @Test
    @XrayTest(summary = "addition of two numbers", description = "tests the sum of two positive integers")
    public void CanAddNumbers()

The problem is that allure uses the description field from the @Test annotation as it seems. I would need to research and reflect a bit
Well, maybe this should be a discussion to pursue on the proper GitHub repository to not pollute this project :)
Feel free to open an issue explaining the need on https://github.com/Xray-App/xray-testng-extensions and eventually some proposal on how to address it :)

vlad230 · 2022-09-14T11:38:10Z

Hi @bitcoder,

I've created the issue on your original project since issues are disabled on the app.getxray one.

Thanks a lot for publishing the library, looking forward to seeing it available on maven central :)

I have read the docs on the new plugin and it should be OK for me to use for now :)

bitcoder · 2022-09-14T12:17:49Z

Oh, I fixed that Issues "issue" :)
Can you please copy paste the issue to the project https://github.com/Xray-App/xray-testng-extensions ? Thanks in advance
I will probably remove the old project

vlad230 · 2022-09-14T12:21:10Z

@bitcoder added the issue in the new project here: Xray-App/xray-testng-extensions#1

Any ideas how long does it usually take for the published library to be available? https://search.maven.org/search?q=g:app.getxray

bitcoder · 2022-09-14T13:18:43Z

it can be a few hours @vlad230 per Maven Central documentation

This was referenced Feb 2, 2018

#1673 enable generateTestResultAttributes using Maven #1681

Closed

#1673 enable generateTestResultAttributes using Maven #1682

Closed

juherr added Feature: report Runner: maven labels Feb 2, 2018

krmahadevan closed this as completed Feb 4, 2018

Yamuna-Rani mentioned this issue Nov 13, 2019

Couldn't set attributes in testng result xml file #2192

Closed

vlad230 mentioned this issue Sep 14, 2022

Use details from the @Test TestNG annotation if present (e.g. description) Xray-App/xray-testng-extensions#1

Closed

pothurajtharun mentioned this issue Feb 17, 2024

How to enable generateTestResultAttributes programatically using Java ? #3070

Closed

7 tasks

how to enable generateTestResultAttributes using Maven #1673

how to enable generateTestResultAttributes using Maven #1673

Comments

bitcoder commented Jan 26, 2018 • edited by juherr Loading

TestNG Version

Expected behavior

Actual behavior

Is the issue reproductible on runner?

Test case sample

krmahadevan commented Jan 26, 2018

bitcoder commented Jan 26, 2018

krmahadevan commented Jan 26, 2018

bitcoder commented Jan 30, 2018

krmahadevan commented Jan 30, 2018

krmahadevan commented Jan 30, 2018

Kanaduchi commented Feb 1, 2018 • edited Loading

krmahadevan commented Feb 2, 2018

Kanaduchi commented Feb 2, 2018 • edited Loading

missedone commented Feb 2, 2018

krmahadevan commented Feb 4, 2018 • edited Loading

krmahadevan commented Feb 4, 2018

missedone commented Feb 4, 2018

juherr commented Feb 4, 2018

Tibor17 commented Feb 4, 2018

juherr commented Feb 4, 2018

Tibor17 commented Feb 4, 2018 • edited Loading

krmahadevan commented Feb 5, 2018 • edited Loading

krmahadevan commented Feb 5, 2018

bitcoder commented Feb 5, 2018

vivekthangathurai commented May 14, 2018 • edited by krmahadevan Loading

krmahadevan commented May 14, 2018

vivekthangathurai commented May 14, 2018

Tibor17 commented May 14, 2018 via email

ssoban commented Nov 13, 2019 • edited by juherr Loading

juherr commented Nov 13, 2019

bitcoder commented Nov 13, 2019

Tibor17 commented Nov 13, 2019

ChathaKiran commented Mar 11, 2020 • edited by krmahadevan Loading

krmahadevan commented Mar 12, 2020

vlad230 commented Sep 13, 2022 • edited Loading

bitcoder commented Sep 13, 2022

vlad230 commented Sep 14, 2022 • edited Loading

bitcoder commented Sep 14, 2022

vlad230 commented Sep 14, 2022

bitcoder commented Sep 14, 2022

vlad230 commented Sep 14, 2022

bitcoder commented Sep 14, 2022

bitcoder commented Jan 26, 2018 •

edited by juherr

Loading

Kanaduchi commented Feb 1, 2018 •

edited

Loading

Kanaduchi commented Feb 2, 2018 •

edited

Loading

krmahadevan commented Feb 4, 2018 •

edited

Loading

Tibor17 commented Feb 4, 2018 •

edited

Loading

krmahadevan commented Feb 5, 2018 •

edited

Loading

vivekthangathurai commented May 14, 2018 •

edited by krmahadevan

Loading

ssoban commented Nov 13, 2019 •

edited by juherr

Loading

ChathaKiran commented Mar 11, 2020 •

edited by krmahadevan

Loading

vlad230 commented Sep 13, 2022 •

edited

Loading

vlad230 commented Sep 14, 2022 •

edited

Loading