Quarkus Platform should not ban test scope in endorsed third party test projects

[ppalaga]

This is a followup of apache/camel-quarkus#35 cc @lburgazzoli

Background: @aloubyansky 's proposal https://github.com/aloubyansky/moving-camel-out-of-quarkus/blob/master/it/camel-servlet/pom.xml#L26 how endorsed third party extension tests should look like bans <scope>test</scope> in integration test projects. This is motivated by the fact that @aloubyansky wants to be able to run these tests inside the planned Quarkus Platform build. Dependencies with <scope>test</scope> would not be resolved properly there.

We (camel-quarkus team) accepted the ban of test scope for now. However, we would not like this to be a definitive solution, because this makes our integration tests non-idiomatic and thus non-usable as quickstarts. It would be nice if Quarkus team could come with a solution that allows us keeping to use <scope>test</scope> in camel-quarkus.

[lburgazzoli]

@aloubyansky does the removal of the test scope eventually impact the build time as test dependencies leak into the compile classpath ?

[aloubyansky]

Right, they will end up in the lib dir.

[aloubyansky]

That's a tricky one. The issue is that the test deps should be loadable by the surefire plugin. For that to work there are a few options:

1. the way it is now, i.e. test scope should be banned for test artifacts;
2. the test scope deps should be duplicated in the platform testsuite pom;
3. some kind of classloading hack.

Personally, I find 2 is worse than 1. Test artifacts are supposed to be used in the testsuite anyway. So, using anything but test scope shouldn't be a big deal. This is how maven dependency resolution works. The issue is not specific to Quarkus, it's how external tests are supposed to be integrated in general, afaiu.

3 is not nice by definition, although that depends on the impl, I guess. Here is a hack that appears to work:

• add a tmp dir to the surefire plugin as an additional classpath element;
• add another bootstrap maven plugin (executed before surefire) which resolves the missing test deps and unpacks them into that tmp dir.

I also tried another hack - modifying the surefire configuration using Maven API during the build which didn't work (I guess for good).

[aloubyansky]

The way I see it, the Maven way to import test JARs applied to Quarkus suggests separation of the test application (the app used in the tests) and the tests themselves by moving them into their own modules. So the test app would be developed in one module and the tests - in a different one which would depend on the test app artifact.
When Maven resolves dependencies of a project it does not include test deps of its direct deps, even if the direct dependency is a test jar. Which means if a test jar is to be imported by other projects, its deps that are required during testing shouldn't be test scoped.
It doesn't seem great, I'm just trying to figure out the pure Maven model to do this.

[ppalaga]

Thanks for the analysis, @aloubyansky . Let me check if we can hack our build process to modify the test jars as you need. Because I do not like the idea to have separate test modules on our side, I'd write a piece of groovy that generates a pom.xml for the tests jar as-if in a separate module and installs/deploys it together with the test jar under the coordinates of the synthetic module.

[lburgazzoli]

I'm a little worried to have a script to workaround this issue on an external extension side for a number of reasons:

1. each extension need to implement the very same workaround
2. each extension need to maintain the script and be sure it is compatible with quarkus
3. is is fragile

A better solution would be to have support from the quarkus tooling side so each extension can just use it and what happens it controlled by quarkus so you can roll-out fixes or implement additional feature easily

[ppalaga]

Thinking of it again, the generation of the POM for the synthetic tests module and installing it to the local maven repo [1] could be also run on the Quarkus side. That would indeed solve the problems named by @lburgazzoli above. WDYT, @aloubyansky ?

[1] apache/camel-quarkus#140

[maxandersen]

related to https://github.com/quarkusio/quarkus-platform/issues/7

[ppalaga]

Yes, quarkusio/quarkus-platform#7 is our proposal, how the current issue should be solved so closing this one.

[maxandersen]

reopening as by accident issues were enabled on platform - no need to spread that thin across.

here is comment from @ppalaga on latest progress with title "Run third party integration tests ":

@aloubyansky agreed to try this proposal. cc @maxandersen

Third party side:

• The third party projects are required to deploy standard test-jars for their integration tests to a public Maven repo, e.g. Central.

Quarkus Platform side:

• The third party test-jars will be transformed using rpkgtests-maven-plugin inside a dedicated maven module (let's call it "rpkgtests"). The rpkgtests module will be just one and it will cover all test-jars from all third parties. Here is an example where two test-jars (jdbc and csv) from Camel Quarkus are transformed:
  https://github.com/ppalaga/rpkgtests-maven-plugin/blob/master/src/it/rpkg-and-run-tests/rpkgtests/pom.xml#L28-L68

• Each 3rd party test jar will have a dedicated module in Quarkus Platform. So in our example, we have two run-tests-* modules: one for csv and one for jdbc: https://github.com/ppalaga/rpkgtests-maven-plugin/tree/master/src/it/rpkg-and-run-tests

• Each run-tests-* module will depend on the rpkgtests module excluding all its deps https://github.com/ppalaga/rpkgtests-maven-plugin/blob/master/src/it/rpkg-and-run-tests/run-tests-csv/pom.xml#L29-L39 This is just to ensure that the test-jars are transformed before they are resolved as dependencies.

• Each run-tests-* module will depend on the given test-jars jar (the test application) and on the transformed test-jar having the -rpkgtests suffix https://github.com/ppalaga/rpkgtests-maven-plugin/blob/master/src/it/rpkg-and-run-tests/run-tests-csv/pom.xml#L40-L50

• To make the Quarkus tooling boot the test app, the following two tweaks are needed
  ** In Quarkus: Support running tests from third party test-jars transformed using #4508
  ** In the test module itself: there must me at least one class in
  src/test/java or there must be a file in src/test/resources

[aloubyansky]

There was an adjustment in Quarkus 0.25.0 to allow the integrating the tests described above.