Add new state-based method for deciding if tests should run in a new classloader

[holly-cummins]

In the current test model, we order tests, and then, at the point of execution, decide if a new classloader is needed. In the post-https://github.com/orgs/quarkusio/projects/30 world, we decide if a new classloader is needed when we load the tests, which is before ordering.

Since we call it before the orderer is called, there’s no notion of a sequence. This means moving from a sequence-based calculation to a state-based calculation. (I know the current parameter is literally called “state,” but really what’s being looked at is the sequence.)

I've implemented this by generating a uniqueness key for every test.

But this new code isn't used anywhere

This code isn't used yet, but once #34681 drops, it will be. I'm being optimistic, and starting to pull incremental chunks.

My new method could be used in the implementation of testResourcesRequireReload. I didn’t yet, for two reasons:

• I was a bit worried that creating strings and then comparing would be less performant than the ‘raw’ comparisons done in that method
• If I replace testResourcesRequireReload, I wouldn’t have anything to compare behaviour against in unit tests :) I guess this doesn’t actually matter since there are unit tests and as long as the unit tests pass, we can assume the new behaviour is good, but it made me hesitate.

I think this method perhaps could and should also be called in the junit orderer, because it can be used to group and sort test classes. That would ensure consistency across all three paths – the ordering in the orderer, the grouping in the FacadeClassLoader (coming soon ™), and the reload decision in the test extensions. I didn’t do this yet, just to keep things incremental and get feedback first.

If it’s used on the orderer, it would also needs to look at the profile (and not just the profile resources), but that’s doable, as long as we didn't accidentally conflate it with the reload decision and reload for every profile.

[quarkus-bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 4da51b3.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

---

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17

📦 extensions/resteasy-reactive/rest-client/deployment

✖ io.quarkus.rest.client.reactive.timeout.BuilderReadTimeoutTest.shouldNotTimeoutOnFastResponse - History

• The timeout period of 1000ms has been exceeded while executing GET /fast for server null - jakarta.ws.rs.ProcessingException

jakarta.ws.rs.ProcessingException: The timeout period of 1000ms has been exceeded while executing GET /fast for server null
	at org.jboss.resteasy.reactive.client.impl.InvocationBuilderImpl.unwrap(InvocationBuilderImpl.java:212)
	at org.jboss.resteasy.reactive.client.impl.InvocationBuilderImpl.method(InvocationBuilderImpl.java:313)
	at io.quarkus.rest.client.reactive.timeout.BuilderReadTimeoutTest$Client$$QuarkusRestClientInterface.fast(Unknown Source)
	at io.quarkus.rest.client.reactive.timeout.BuilderReadTimeoutTest.shouldNotTimeoutOnFastResponse(BuilderReadTimeoutTest.java:53)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at io.quarkus.test.QuarkusUnitTest.runExtensionMethod(QuarkusUnitTest.java:513)
	at io.quarkus.test.QuarkusUnitTest.interceptTestMethod(QuarkusUnitTest.java:427)

---

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging-kafka/deployment

✖ io.quarkus.smallrye.reactivemessaging.kafka.deployment.testing.KafkaDevServicesContinuousTestingWorkingAppPropsTestCase.testContinuousTestingScenario3 - History

• io.quarkus.builder.BuildException: Build failure: Build failed due to errors [error]: Build step io.quarkus.redis.deployment.client.DevServicesRedisProcessor\#startRedisContainers threw an exception: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image docker.io/redis:7 at io.quarkus.redis.deployment.client.DevServicesRedisProcessor.startRedisContainers(DevServicesRedisProcessor.java:124) at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733) at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856) - java.lang.RuntimeException

java.lang.RuntimeException: 
io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.redis.deployment.client.DevServicesRedisProcessor#startRedisContainers threw an exception: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image docker.io/redis:7
	at io.quarkus.redis.deployment.client.DevServicesRedisProcessor.startRedisContainers(DevServicesRedisProcessor.java:124)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)

📦 integration-tests/grpc-hibernate

✖ com.example.grpc.hibernate.BlockingRawTest.shouldAdd - History

• Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:975)
	at com.example.grpc.hibernate.BlockingRawTestBase.shouldAdd(BlockingRawTestBase.java:59)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)

---

⚙️ JVM Tests - JDK 17 Windows

📦 extensions/micrometer/deployment

✖ io.quarkus.micrometer.deployment.binder.VertxHttpClientMetricsTest.testWebClientMetrics - History

• event executor terminated - java.util.concurrent.RejectedExecutionException

java.util.concurrent.RejectedExecutionException: event executor terminated
	at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:934)
	at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353)
	at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346)
	at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:836)
	at io.netty.util.concurrent.SingleThreadEventExecutor.execute0(SingleThreadEventExecutor.java:827)
	at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:817)
	at io.vertx.core.impl.EventLoopExecutor.execute(EventLoopExecutor.java:35)

📦 integration-tests/opentelemetry-quickstart

✖ io.quarkus.it.opentelemetry.OpenTelemetryDisabledTest.buildTimeDisabled - History

• Condition with Lambda expression in io.quarkus.it.opentelemetry.OpenTelemetryDisabledTest was not fulfilled within 200 milliseconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in io.quarkus.it.opentelemetry.OpenTelemetryDisabledTest was not fulfilled within 200 milliseconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:975)
	at io.quarkus.it.opentelemetry.OpenTelemetryDisabledTest.buildTimeDisabled(OpenTelemetryDisabledTest.java:29)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)