Camel with opentelemetry - Context in storage not the expected context, Scope.close was not called correctly

[brianmolinaspring]

Bug description

Describe the bug

Context in storage not the expected context, Scope.close was not called correctly

Expected behavior

Not receiving the warning

Actual behavior

Warning : Context in storage not the expected context, Scope.close was not called correctly

How to Reproduce?

when using version [3.0.0-M1] onwards it happens, dependency used:

org.apache.camel.quarkus camel-quarkus-opentelemetry 3.7.0Output of uname -a or ver

No response

Output of java -version

17

Quarkus version or git rev

3.2.10.Final

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

How can I remove this warning since I am using a log collection and I do not want to show those warnings

[jamesnetherton]

Have you tried with the latest Quarkus 3.7.3 release?

I see some open issues in Quarkus documenting a similar problem.

quarkusio/quarkus#25102
quarkusio/quarkus#35686

[brianmolinaspring]

hi @jamesnetherton again me jjeje, I have not tried with that version, I am using 3.2.10.Final, because I understand it is the one that should be used for prod, everything that ends in Final is the most stable according to the documentation but if you tell me that I can use 3.7.3 release, can I do it, what do you think I can use this to take my microservice from camel to prod?

[jamesnetherton]

everything that ends in Final is the most stable according to the documentation

Not quite. The .Final suffix has actually been removed in recent Quarkus releases. However Quarkus & Camel have Long Term Support (LTS) releases. Currently 3.2.x and soon to be 3.8.x.

[brianmolinaspring]

ok @jamesnetherton , then I upgrade to version 3.7.3?

[jamesnetherton]

It depends what your requirements are. Take a read through this:

https://quarkus.io/blog/lts-releases/

Quarkus 3.8.0 will be released soon, which is the next LTS release.

[brianmolinaspring]

I went up to 3.7.3 but it still shows the warning, there is no way to avoid it, I mean it's just a warning, can it bring us some performance issue?

09:42:18 WARN traceId=bb01824666be557e3835344115fe0621, parentId=a387ba815117ea10, spanId=f48fca554eeff53a, sampled=true [io.qu.op.ru.QuarkusContextStorage] (vert.x-worker-thread-1) Context in storage not the expected context, Scope.close was not called correctly. Details: OTel context before: {spanId=f48fca554eeff53a, traceId=bb01824666be557e3835344115fe0621, sampled=true}. OTel context toAttach: {spanId=a387ba815117ea10, traceId=bb01824666be557e3835344115fe0621, sampled=true, parentId=f48fca554eeff53a}

[jamesnetherton]

it's just a warning

Yeah, but something is likely going wrong somewhere. Have you checked that the all of the expected traces / spans are delivered to the OpenTelemetry collector?

[brianmolinaspring]

use this dependency

org.apache.camel.quarkus
camel-quarkus-opentelemetry
3.7.0

and then in my properties

quarkus.opentelemetry.enabled=true
quarkus.opentelemetry.tracer.exporter.otlp.endpoint=http://192.168.1.1:4317

That is enough to send the traces to Jaeger and he receives it correctly. Be careful, notice something strange in the dependency, one version less, for example.
2.16.0 does not present that warning, but I see that it does not show me all the span, what I mean is that I only see the span of the POST /customer message but it does not show me the other span, I suppose it is incompatibility between versions.

What are you suggesting ?

[jamesnetherton]

What are you suggesting ?

That whatever is going wrong to cause the WARN message, may affect your application tracing. Hence, why checking the results would be a good idea.

[brianmolinaspring]

@jamesnetherton I downgraded to 3.2.4.Final, and the warning disappeared, do you think I should keep this version? And regarding the warning, could it be that the new versions have some irregularity with these warnings?

Question, can the scope be closed manually and thus the warning is avoided? Because to export the traces I see that it is not necessary to add any logic to the source code, but if you tell me that it is possible, could you give me instructions?

[jamesnetherton]

I downgraded to 3.2.4.Final, and the warning disappeared, do you think I should keep this version

Not really, because 3.2.4 is affected by multiple CVEs. If you want to stay on 3.2.x then use 3.2.10.

Because to export the traces I see that it is not necessary to add any logic to the source code

Yes, I know that. What I'm saying is that whatever is going on to cause the WARN message, could (it may not) impact your tracing results.

This issue needs some deeper investigation. If you can provide an example application that reproduces the problem, it'd help resolve it. Camel Quarkus OpenTelemetry depends on Quarkus OpenTelemetry, so we may be dependent on Quarkus to get a fix.

[brianmolinaspring]

@jamesnetherton You can clone the quarkus camel example that they put in the documentation.

git clone https://github.com/apache/camel-quarkus-examples.git

$ cp -r camel-quarkus-examples/rest-json .
$ cd rest-json

and add this to the pom

  <dependency>
        <groupId>org.apache.camel.quarkus</groupId>
        <artifactId>camel-quarkus-opentelemetry</artifactId>
    </dependency>

and properties

quarkus.application.name=test-fruist
quarkus.opentelemetry.enabled=true
quarkus.opentelemetry.tracer.exporter.otlp.endpoint=http://192.168.1.1:4317

if we try through the browser

http://localhost:8080/fruits

we can see the warning in the logs

2024-02-16 10:46:39,588 WARN [io.qua.ope.run.QuarkusContextStorage] (vert.x-worker-thread-1) Context in storage not the expected context, Scope.close was not called correctly. Details: OTel context before: {spanId=3cd2ab2cf982a2ff, traceId=5117cbcdac3588c36ff01e5b4f704873, sampled=true}. OTel context toAttach: {spanId=781147065bf56c38, traceId=5117cbcdac3588c36ff01e5b4f704873, sampled=true, parentId=3cd2ab2cf982a2ff}

It's okay like that ? Would you help me with that?

[brianmolinaspring]

@jamesnetherton One question, if I can't use 3.2.4, what else do you recommend that is a lower version and stable? I am working with a ms and I need to upload it between the different environments but without that warning we already use eFK to collect the logs and this warning would add this detail to the efk logs that are not necessary.

[brianmolinaspring]

@jamesnetherton Could you see something about the example?

[brianmolinaspring]

@jamesnetherton I managed to hide the message using a quarkus filter

package com.ms.bss.utilities;

import java.util.logging.Filter;
import java.util.logging.LogRecord;

import io.quarkus.logging.LoggingFilter;
import org.eclipse.microprofile.config.inject.ConfigProperty;

@LoggingFilter(name = "my-filter")
public final class TestFilter implements Filter {

private final String part;

public TestFilter(@ConfigProperty(name = "my-filter.part") String part) {
    this.part = part;
}

@Override
public boolean isLoggable(LogRecord record) {
    return !record.getMessage().contains(part);
}

}

in properties:

my-filter.part=Scope.close
quarkus.log.console.filter=my-filter

As mentioned in the other buses reported, the warning is harmless, is it okay to hide the warning while adjusting a new version?

I plan to take this version of Quarkus 3.2.10.Final to production, is that okay with you?

[jamesnetherton]

take this version of Quarkus 3.2.10.Final to production

Yes, that is the latest LTS release so should be good.

Newer LTS releases for Quarkus 3.8.0 + Camel 4.4.0 will be out at the end of the month.

[brianmolinaspring]

@jamesnetherton thanks. Do you see well the filter to take it to prod like this?