Don't apply JUnit reports plugin by default

[pkoenig10]

The JUnit reports plugin has a couple of problems.

It creates test reports for Gradle tasks

This is very strange. Test reports are intended to be for tests, not for arbitrary tasks. This is confusing in other services that integrate with Gradle test reports (like CircleCI) because it appears that tests have failed when they haven't.

In addition, this causing confusing output when tests are sharded across multiple CI nodes. For true test failures, the failure is only reported once (by the node that ran the test). But Gradle task failures are reported by every node. This leads to confusing output as shown in the screenshot below. We have 5 CI nodes and these errors are reported 5 times (the screenshot just shows 2).

This also creates a ton of garbage output locally. The build.xml files are never deleted and this plugin just happily keeps adding 1 until it finds a file name that doesn't exist. Here's an example from one of my local projects:

It stores test reports in a non-standard place

By default Gradle puts test reports in the following places:

• JUnit XML test reports in {projectDir}/build/test-results/test
• HTML test reports in {projectDir}/build/reports/tests/test

But the test reporting plugin moves the JUnit XML test reports to the root project in a custom directory:

• JUnit XML test reports in {rootProjectDir}/build/junit-reports/junit/{projectName}/test
• HTML test reports in {projectDir}/build/reports/tests/test

There's no good reason to do this. It just makes it hard for devs to understand how our infrastructure works. This is especially confusing because documentation for other tools often references the standard behavior. For example, in the CircleCI docs.

---

I don't anticipate this being a disruptive change for a couple reasons:

• Our standard CircleCI configs look for files using the regex .*/build/.*TEST.*xml, which will find JUnit XML test reports even in the standard output location.
• No one appears to be using custom report. Searching internally for the junitReports or junitTaskResults Gradle extension config, I find just a single use.

I'm strongly inclined to just remove this JUnit reporting plugin altogether. It appears to be wildly over-engineered to support arbitrary/custom reports, unnecessarily messes with standard Gradle conventions, and is confusingly called "junit reports" even though these reports are definitely not related to JUnit.

[changelog-app]

Generate changelog in changelog/@unreleased

Type
See change types. Select one:

• Feature
• Improvement
• Fix
• Break
• Deprecation
• Manual task
• Migration

Description

The JUnits reports plugin is no longer applied by default. Test reports now use the standard output locations from Gradle conventions.

Check the box to generate changelog(s)

• Generate changelog entry

[iamdanfox]

I'm fine with this - after the CircleCI 3 upgrade the UI has changed to describe these XML items as 'tests' which I agree is now misleading.

[svc-autorelease]

Released 4.154.0

[pkoenig10]

Would you be open to just removing this code entirely? Or do you think it still has value?

[pkoenig10]

Put up a PR for this in #2356 if it's something we want to do.