Tracing: SSE is overwriting "spans" on opentracing + resteasy-reactive

[pjgg]

Describe the bug

Quarkus: Upstream
Extensions: quarkus-resteasy-reactive, quarkus-smallrye-opentracing, quarkus-rest-client-reactive

If I request some (server-sent events) data from a Service A that use a Service B, I would expect one trace with two Spans (one per service)., however I am getting a trace with a single span (the latest one).

Note: used to work with quarkus-resteasy-mutiny

Trace:

{
  "data": [
    {
      "traceID": "f3623e0a8ba26c4f",
      "spans": [
        {
          "traceID": "f3623e0a8ba26c4f",
          "spanID": "1589802d29578d45",
          "flags": 1,
          "operationName": "GET",
          "references": [
            {
              "refType": "CHILD_OF",
              "traceID": "f3623e0a8ba26c4f",
              "spanID": "f3623e0a8ba26c4f"
            }
          ],
          "startTime": 1669651777429194,
          "duration": 45119,
          "tags": [
            {
              "key": "http.status_code",
              "type": "int64",
              "value": 200
            },
            {
              "key": "component",
              "type": "string",
              "value": "jaxrs"
            },
            {
              "key": "span.kind",
              "type": "string",
              "value": "client"
            },
            {
              "key": "http.url",
              "type": "string",
              "value": "http://localhost:1101/server-sent-events-pong"
            },
            {
              "key": "peer.hostname",
              "type": "string",
              "value": "localhost"
            },
            {
              "key": "peer.port",
              "type": "int64",
              "value": 1101
            },
            {
              "key": "http.method",
              "type": "string",
              "value": "GET"
            },
            {
              "key": "internal.span.format",
              "type": "string",
              "value": "jaeger"
            }
          ],
          "logs": [],
          "processID": "p1",
          "warnings": [
            "invalid parent span IDs=f3623e0a8ba26c4f; skipping clock skew adjustment"
          ]
        }
      ],
      "processes": {
        "p1": {
          "serviceName": "pingpong",
          "tags": [
            {
              "key": "hostname",
              "type": "string",
              "value": "fedora"
            },
            {
              "key": "ip",
              "type": "string",
              "value": "192.168.68.106"
            },
            {
              "key": "jaeger.version",
              "type": "string",
              "value": "Java-1.8.1"
            }
          ]
        }
      },
      "warnings": null
    }
  ],
  "total": 0,
  "limit": 0,
  "offset": 0,
  "errors": null
}

Expected behavior

One trace with two Spans, one per HTTP service that is involved in the transaction.

Actual behavior

Looks like the second API overwrites the first span.

How to Reproduce?

git clone --branch QUARKUS-2303 [email protected]:pjgg/quarkus-test-suite.git

Move on to module monitoring/opentracing-reactive-grpc

mvn clean verify -Dit.test=ServerSentEventsTraceOpentracingIT

Classes that are involved on this scenario:
io.quarkus.ts.monitoring.opentracing.reactive.grpc.ping.ServerSentEventsPingResource --> Service A
io.quarkus.ts.monitoring.opentracing.reactive.grpc.ping.clients.ServerSentEventsPongClient --> http Client of serviceB
io.quarkus.ts.monitoring.opentracing.reactive.grpc.pong.ServerSentEventsPongResource --> Service B

ServiceA --> HttpClient --> Service B

Output of uname -a or ver

Linux fedora 6.0.9-200.fc36.x86_64

Output of java -version

openjdk 17.0.5 2022-10-18

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

[quarkus-bot]

/cc @FroMage, @Ladicek, @brunobat, @geoand, @radcortez, @stuartwdouglas

[pjgg]

Also there is a warning in Jaeger on the trace/span

invalid parent span IDs=554ced9e6c91a2d3; skipping clock skew adjustmen

Is like, no span was created on SSE endpoints and the only span that was created was related to the HttpClient that is in the middle.

[brunobat]

@pjgg OpenTracing is no longer supported and users are incentivised to use its sucessor OpenTelemetry instead.
Please see: https://quarkus.io/blog/quarkus-observability-roadmap-2023/
And https://quarkus.io/guides/opentelemetry
There were context issues that have already been solved with the current OpenTelemetry implementation and could explain the reported behaviour.

[pjgg]

From my side we can close this issue :)
Just for the records the legacy quarkus-smallrye-opentracing is working as expected with quarkus-resteasy-mutiny (both deprecated in favor of quarkus-opentelemetry and quarkus-resteasy-reactive)