Jakarta - EE 9 - MP TCK - Failures in REST Client TCK

[gsmet]

We have several failures:

✖️ org.eclipse.microprofile.rest.client.tck.timeout.TimeoutBuilderIndependentOfMPConfigTest.testConnectTimeout

✖️ org.eclipse.microprofile.rest.client.tck.timeout.TimeoutTest.testConnectTimeout

✖️ org.eclipse.microprofile.rest.client.tck.timeout.TimeoutViaMPConfigTest.testConnectTimeout

✖️ org.eclipse.microprofile.rest.client.tck.timeout.TimeoutViaMPConfigTest.testReadTimeout

✖️ org.eclipse.microprofile.rest.client.tck.timeout.TimeoutViaMPConfigWithConfigKeyTest.testConnectTimeout

✖️ org.eclipse.microprofile.rest.client.tck.timeout.TimeoutViaMPConfigWithConfigKeyTest.testReadTimeout

[quarkus-bot]

/cc @michalszynkiewicz

[michalszynkiewicz]

CC @Sgitario
Also, @xstefank I think you played with these tests some time ago, maybe you'd like to take a look

[Sgitario]

I had a look into these issues and I could see the following errors:

[ERROR] Failures:
[ERROR]   TimeoutBuilderIndependentOfMPConfigTest.testConnectTimeout » Test
Expected exception of type class jakarta.ws.rs.ProcessingException but got java.lang.AssertionError: Elapsed time expected under 5030ms, but was 10096ms. expected [true] but found [false]
[ERROR]   TimeoutTest.testConnectTimeout » Test
Expected exception of type class jakarta.ws.rs.ProcessingException but got java.lang.AssertionError: Elapsed time expected under 5030ms, but was 10082ms. expected [true] but found [false]
[ERROR]   TimeoutViaMPConfigTest.testConnectTimeout » Test
Expected exception of type class jakarta.ws.rs.ProcessingException but got java.lang.AssertionError: Elapsed time expected under 7030ms, but was 14103ms. expected [true] but found [false]
[ERROR]   TimeoutViaMPConfigTest.testReadTimeout » Test
Expected exception of type class jakarta.ws.rs.ProcessingException but got java.lang.AssertionError: Elapsed time expected under 7030ms, but was 14016ms. expected [true] but found [false]
[ERROR]   TimeoutViaMPConfigWithConfigKeyTest.testConnectTimeout » Test
Expected exception of type class jakarta.ws.rs.ProcessingException but got java.lang.AssertionError: Elapsed time expected under 7030ms, but was 14034ms. expected [true] but found [false]
[ERROR]   TimeoutViaMPConfigWithConfigKeyTest.testReadTimeout » Test
Expected exception of type class jakarta.ws.rs.ProcessingException but got java.lang.AssertionError: Elapsed time expected under 7030ms, but was 14017ms. expected [true] but found [false]
[INFO]
[ERROR] Tests run: 8, Failures: 6, Errors: 0, Skipped: 0

It's suspicious that the times are almost the double number expected, but maybe this is not relevant.
However, I could find why these tests started failing. Though I confirmed the relevant properties are passed to the builder property. So I will wait for @xstefank, just in case he knows more and if not, we should probably ask the Resteasy team.

More context:

• the RESTEasy dependencies were bumped from 4.7.7 to 6.0.3 (but I could not find any relevant change).
• the RESTeasy TCK is using Wiremock which is incompatible with Jakarta 9. I tried to use wiremock jr8 standalone but it caused more troubles. Having said that, I don't think this is related to these test failures.

[gsmet]

@Sgitario I think Wiremock is working correctly as I spent time making sure each TCK was at least working properly (i.e. we don't have a gazillion of tests failing).

RESTEasy was updated, yes (and will be updated again as soon as we switch to EE 10).

[gsmet]

Also, QuarkusRestClientBuilder had to change a bit to be compatible with the new version.

[xstefank]

hi, so yes, I've run into this problem on my machine, but I assumed it was just slow, a lot of going on always on my computer. I override the config like this - -Dorg.eclipse.microprofile.rest.client.tck.timeoutCushion=1500 - might be slightly different for CI but it worked. I didn't have time to investigate deeper as I was fixing something else.

[Sgitario]

I tried to reproduce this issue and indeed the timeouts are working fine (in both Java 17, 11 and using the rest client builder and the injected client via CDI), so it might be related to the test suite.

[gsmet]

FWIW, the TCK issue is reproducible always locally on my machine with JDK 11. For now, I will disable them: #29730 .

[rsvoboda]

Hi @gsmet, @xstefank, @geoand. org.eclipse.microprofile.rest.client.tck.timeout.Timeout* tests are disabled in the default config, but executed in separate one with <jdk.net.usePlainSocketImpl>true</jdk.net.usePlainSocketImpl> set.

https://github.com/quarkusio/quarkus/blob/main/tcks/microprofile-rest-client/pom.xml#L202

The goal of this issue is to make them pass in the default config without usePlainSocketImpl?

[geoand]

This was added in #19559

[xstefank]

I guess, I still plan to look into what this is about

[radcortez]

TCKs are passing.