Running a shutdown task failed: Provider io.smallrye.faulttolerance.propagation.ContextPropagationExecutorFactory not a subtype

[rsvoboda]

I circled back to the https://github.com/quarkus-qe/beefy-scenarios to try it with current Quarkus master and I noticed stacktrace in the console, tests do not fail though.

Stacktrace is around faulttolerance, so @Ladicek / @michalszynkiewicz could you please take a look ?

Steps to reproduce:

git clone https://github.com/quarkus-qe/beefy-scenarios.git
cd beefy-scenarios
mvn clean verify -fae -pl 101-javaee-like-getting-started/

Details

[INFO] Running io.quarkus.qe.core.JavaEELikeQuarkusBundleTest
2020-02-12 13:31:02,611 WARN  [io.qua.agr.dep.AgroalProcessor] (build-36) Agroal dependency is present but no driver has been defined for the default datasource
2020-02-12 13:31:05,133 INFO  [io.quarkus] (main) Quarkus 999-SNAPSHOT started in 3.685s. Listening on: http://0.0.0.0:8081
2020-02-12 13:31:05,134 INFO  [io.quarkus] (main) Profile test activated.
2020-02-12 13:31:05,134 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, hibernate-validator, jaeger, narayana-jta, resteasy, resteasy-jsonb, scheduler, security, servlet, smallrye-context-propagation, smallrye-fault-tolerance, smallrye-health, smallrye-metrics, smallrye-openapi, smallrye-opentracing, swagger-ui, undertow-websockets]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.82 s - in io.quarkus.qe.core.JavaEELikeQuarkusBundleTest
[INFO] Running io.quarkus.qe.core.JavaEELikeHealthCheckTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.556 s - in io.quarkus.qe.core.JavaEELikeHealthCheckTest
2020-02-12 13:31:07,593 ERROR [io.qua.run.StartupContext] (main) Running a shutdown task failed: java.util.ServiceConfigurationError: io.smallrye.faulttolerance.ExecutorFactory: Provider io.smallrye.faulttolerance.propagation.ContextPropagationExecutorFactory not a subtype
	at java.util.ServiceLoader.fail(ServiceLoader.java:239)
	at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
	at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
	at io.smallrye.faulttolerance.ExecutorProvider.executorProvider(ExecutorProvider.java:83)
	at io.smallrye.faulttolerance.ExecutorProvider.setUp(ExecutorProvider.java:48)
	at io.smallrye.faulttolerance.ExecutorProvider_Bean.create(ExecutorProvider_Bean.zig:398)
	at io.smallrye.faulttolerance.ExecutorProvider_Bean.create(ExecutorProvider_Bean.zig:414)
	at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:80)
	at io.quarkus.arc.impl.ComputingCache$CacheFunction.lambda$apply$0(ComputingCache.java:99)
	at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:26)
	at io.quarkus.arc.impl.ComputingCache.getValue(ComputingCache.java:41)
	at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:25)
	at io.smallrye.faulttolerance.ExecutorProvider_Bean.get(ExecutorProvider_Bean.zig:472)
	at io.smallrye.faulttolerance.ExecutorProvider_Bean.get(ExecutorProvider_Bean.zig:65)
	at io.smallrye.faulttolerance.ExecutorProvider_Observer_tearDown_5b12a3f0c93ede4d9438df3ab2b4167b7b4850fa.notify(ExecutorProvider_Observer_tearDown_5b12a3f0c93ede4d9438df3ab2b4167b7b4850fa.zig:71)
	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:282)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:267)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:217)
	at io.quarkus.arc.impl.ArcContainerImpl.shutdown(ArcContainerImpl.java:302)
	at io.quarkus.arc.Arc.shutdown(Arc.java:49)
	at io.quarkus.arc.runtime.ArcRecorder$1.run(ArcRecorder.java:39)
	at io.quarkus.runtime.StartupContext.runAllInReverseOrder(StartupContext.java:69)
	at io.quarkus.runtime.StartupContext.close(StartupContext.java:58)
	at io.quarkus.runner.ApplicationImpl.doStop(ApplicationImpl.zig:527)
	at io.quarkus.runtime.Application.stop(Application.java:170)
	at io.quarkus.runtime.Application.close(Application.java:113)
	at io.quarkus.runner.bootstrap.StartupActionImpl$1.close(StartupActionImpl.java:102)
	at io.quarkus.runner.bootstrap.RunningQuarkusApplicationImpl.close(RunningQuarkusApplicationImpl.java:29)
	at io.quarkus.test.junit.QuarkusTestExtension$1.close(QuarkusTestExtension.java:100)
	at io.quarkus.test.junit.QuarkusTestExtension$ExtensionState.close(QuarkusTestExtension.java:369)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.execution.ExtensionValuesStore.closeAllStoredCloseableValues(ExtensionValuesStore.java:61)
	at org.junit.jupiter.engine.descriptor.AbstractExtensionContext.close(AbstractExtensionContext.java:73)
	at org.junit.jupiter.engine.execution.JupiterEngineExecutionContext.close(JupiterEngineExecutionContext.java:53)
	at org.junit.jupiter.engine.descriptor.JupiterEngineDescriptor.cleanUp(JupiterEngineDescriptor.java:67)
	at org.junit.jupiter.engine.descriptor.JupiterEngineDescriptor.cleanUp(JupiterEngineDescriptor.java:29)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$cleanUp$9(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.cleanUp(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:83)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:142)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:117)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

2020-02-12 13:31:07,594 INFO  [io.quarkus] (main) Quarkus stopped in 0.040s

[Ladicek]

I wonder how come that io.smallrye.faulttolerance.propagation.ContextPropagationExecutorFactory is not a subtype of io.smallrye.faulttolerance.ExecutorFactory. The only thing I can possibly come up with is that they are loaded by different classloaders.

[rsvoboda]

had similar feeling as the error didn't cause test failure

[Ladicek]

Apparently there were some classloading fixes that made it to 1.3.0.Alpha2. Worth a retest.

[Ladicek]

OK, still happens on latest master. I'll take a look if I can figure out something.

[Ladicek]

Looking at the stack trace again, I see it's happening during shutdown: io.quarkus.runtime.Application.close...

[Ladicek]

Looks like during shutdown, the thread context classloader (which is used by the ServiceLoader mechanism) is no longer the Quarkus base runtime classloader, but the system classloader. However, the SRye Fault Tolerance classes are loaded by the Quarkus base runtime classloader.

[Ladicek]

I'll submit a PR with a fix. It's simple, but I have absolutely no idea whatsoever if it's also correct :-)