Support other JUnit 5 test engines / support JUnit 5 platform

[jlink]

The current version of vscode-java-test only supports the Jupiter engine. The JUnit 5 platform, however, provides support for all kind of test engines. Examples are

• jqwik
• KotlinTest
• MAINRUNNER
• Spek
• TestNG

vscode-java-test is (probably) already using the launcher API under the hood. Since the launcher supports all test engines out of the box the effort should be doable without too much effort.

Most of the effort might be done with #683

[jdneo]

I believe if we support @testable, then jqwik will be automatically supported.

Since there might be huge plenty of different kinds of test framework. It's not a good idea to hard code all of the annotations in the search engine.

[jlink]

I see at least two tasks:

• Enable "Run/Debug Tests" for all classes/methods annotated with an annotation that has @Testable as a meta annotation
• Enable "Run/Debug Tests" for all classes that have methods or inner classes that are testable

The second issue is important to allow nesting and collections of test suites. Several engines offer this functionality.

@jdneo I'd be willing to contribute a PR but instructions in https://github.com/Microsoft/vscode-java-test/blob/master/CONTRIBUTING.md#contributing-fixes were not detailed enough for me to know how to build and test my code changes. Anyone who has time to guide me through it?

[jdneo]

Hi @jlink,

Did you meet any problem setup the environment?

[jlink]

@jdneo
The npm part of the set up seemed to work. My questions are along the lines of:

• How to configure my env. Maven complains that build is platform-dependent but gives no hint about solving that issue.
• Which Maven/NPM task(s) to run for testing - although I haven't found any tests in the sources.
• Which Maven/NPM task to run for linting/checking/building the extension
• How to add the self-build extension in VSCode

[jdneo]

How to configure my env. Maven complains that build is platform-dependent but gives no hint about solving that issue.

If you mean the warning No explicit target runtime environment configuration. Build is platform dependent., you can just ignore it.

Which Maven/NPM task(s) to run for testing - although I haven't found any tests in the sources.

So far we only have some test cases at the client side. you can run them by npm run test.
// Make sure no VS Code instance is opened when running this command

Which Maven/NPM task to run for linting/checking/building the extension

• linting: npm run linting
• building:
  • build the jar: npm run build-plugin. The jars will be generated in the folder server/
  • build the extension installation file: npx vsce@latest package, the output will be a .vsix file

How to add the self-build extension in VSCode

See:

Let me know if you need more information, thanks!

[jlink]

@jdneo Everything worked well so far! I think your enumeration would make a great addition to https://github.com/Microsoft/vscode-java-test/blob/master/CONTRIBUTING.md#contributing-fixes

[jdneo]

Thanks! I'll add them later 😄

[jlink]

@jdneo I managed to get the main thing going. Before opening a PR, though, I'd like to fix some problems with the console launcher params. Is there a way to see stdout/stderr output from the Java part of the extension? All I can find in the web is how to see the Javascript log.

[jdneo]

Please set the setting java.test.log.level to verbose and check the output in the output channel named Java Test Runner. Let me know if this could help.

Thanks.

[jlink]

@jdneo with the settings adpated I get more output but not necessarily what I need.

I started the PR anyway (#980) to give you and others a chance to help me out with the existing problems.

[jdneo]

After some refactoring in #1131, now the test runner can support other JUnit 5 test engines/platforms. (except for those JVM compatible language like Kotlin, which needs the upstream Java Language Server to support it).

So close this issue now.