-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quarkus opentelemetry extension WARNING when exporting to dynatrace #38963
Comments
/cc @brunobat (opentelemetry,tracing), @radcortez (opentelemetry,tracing) |
How can I reproduce this? |
I am thinking how to best reproduce this because it depends on Dynatrace. Basically you need a quarkus application with the open telemetry extension and a dynatrace instance to which quarkus app will connect. My feeling is that the Quarkus opentelemetry extension is considering empty HTTP responses from Dyntrace API as a failure, hence the warning. But it is just a guess |
Could very well be the case, but without being able to test, we can't be sure. One more question: Does Dynatrace show the traces that Quarkus has warned failed? |
@geoand unfortunately I cannot see those traces. Has the Quarkus opentelemetry extension been proven to work with Dynatrace already ? Here is the doc from dynatrace how to integrate with Quarkus https://docs.dynatrace.com/docs/setup-and-configuration/technology-support/application-software/java/quarkus |
That's a question for @brunobat |
Hi @geoand , I came up with a reproducer. First I realized that since the default is grpc, I need to change the exporter protocol in the code to http: Here is the error I get now when starting the Quarkus application: 2024-02-23 14:53:23,599 ERROR [io.qua.run.Application] (Quarkus Main Thread) Failed to start application (with profile [dev]): java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
at io.quarkus.runner.GeneratedMain.main(Unknown Source)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:113)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: jakarta.enterprise.inject.CreationException: Error creating synthetic bean [Le6zQbzkojAYO_OiKIQWJf4lGa4]: jakarta.enterprise.inject.CreationException: Error creating synthetic bean [shT2wDm9BfI89PAXjVYmrqS0z6k]: java.lang.IllegalStateException: Unable to install OTLP Exporter
at io.opentelemetry.api.OpenTelemetry_Le6zQbzkojAYO_OiKIQWJf4lGa4_Synthetic_Bean.doCreate(Unknown Source)
at io.opentelemetry.api.OpenTelemetry_Le6zQbzkojAYO_OiKIQWJf4lGa4_Synthetic_Bean.create(Unknown Source)
at io.opentelemetry.api.OpenTelemetry_Le6zQbzkojAYO_OiKIQWJf4lGa4_Synthetic_Bean.create(Unknown Source)
at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:119)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:38)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:35)
at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:32)
at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
at io.quarkus.arc.impl.ComputingCacheContextInstances.computeIfAbsent(ComputingCacheContextInstances.java:19)
at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:35)
at io.opentelemetry.api.OpenTelemetry_Le6zQbzkojAYO_OiKIQWJf4lGa4_Synthetic_Bean.get(Unknown Source)
at io.opentelemetry.api.OpenTelemetry_Le6zQbzkojAYO_OiKIQWJf4lGa4_Synthetic_Bean.get(Unknown Source)
at io.quarkus.arc.impl.InstanceImpl.getBeanInstance(InstanceImpl.java:325)
at io.quarkus.arc.impl.InstanceImpl.getInternal(InstanceImpl.java:309)
at io.quarkus.arc.impl.InstanceImpl.get(InstanceImpl.java:190)
at io.quarkus.arc.runtime.BeanContainerImpl.beanInstance(BeanContainerImpl.java:26)
at io.quarkus.opentelemetry.runtime.tracing.intrumentation.InstrumentationRecorder.setupVertxTracer(InstrumentationRecorder.java:43)
at io.quarkus.deployment.steps.OpenTelemetryProcessor$setupVertx126580776.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.OpenTelemetryProcessor$setupVertx126580776.deploy(Unknown Source)
... 11 more
Caused by: jakarta.enterprise.inject.CreationException: Error creating synthetic bean [shT2wDm9BfI89PAXjVYmrqS0z6k]: java.lang.IllegalStateException: Unable to install OTLP Exporter
at io.quarkus.opentelemetry.runtime.exporter.otlp.LateBoundBatchSpanProcessor_shT2wDm9BfI89PAXjVYmrqS0z6k_Synthetic_Bean.doCreate(Unknown Source)
at io.quarkus.opentelemetry.runtime.exporter.otlp.LateBoundBatchSpanProcessor_shT2wDm9BfI89PAXjVYmrqS0z6k_Synthetic_Bean.create(Unknown Source)
at io.quarkus.opentelemetry.runtime.exporter.otlp.LateBoundBatchSpanProcessor_shT2wDm9BfI89PAXjVYmrqS0z6k_Synthetic_Bean.create(Unknown Source)
at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:119)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:38)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:35)
at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:32)
at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
at io.quarkus.arc.impl.ComputingCacheContextInstances.computeIfAbsent(ComputingCacheContextInstances.java:19)
at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:35)
at io.quarkus.opentelemetry.runtime.exporter.otlp.LateBoundBatchSpanProcessor_shT2wDm9BfI89PAXjVYmrqS0z6k_Synthetic_Bean.get(Unknown Source)
at io.quarkus.opentelemetry.runtime.exporter.otlp.LateBoundBatchSpanProcessor_shT2wDm9BfI89PAXjVYmrqS0z6k_Synthetic_Bean.get(Unknown Source)
at io.quarkus.arc.impl.Instances.getBeanInstance(Instances.java:115)
at io.quarkus.arc.impl.Instances.listOf(Instances.java:75)
at io.quarkus.arc.impl.ListProvider.get(ListProvider.java:50)
at io.quarkus.arc.impl.ListProvider.get(ListProvider.java:15)
at io.quarkus.opentelemetry.runtime.AutoConfiguredOpenTelemetrySdkBuilderCustomizer_TracerProviderCustomizer_Bean.doCreate(Unknown Source)
at io.quarkus.opentelemetry.runtime.AutoConfiguredOpenTelemetrySdkBuilderCustomizer_TracerProviderCustomizer_Bean.create(Unknown Source)
at io.quarkus.opentelemetry.runtime.AutoConfiguredOpenTelemetrySdkBuilderCustomizer_TracerProviderCustomizer_Bean.create(Unknown Source)
at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:119)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:38)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:35)
at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:32)
at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
at io.quarkus.arc.impl.ComputingCacheContextInstances.computeIfAbsent(ComputingCacheContextInstances.java:19)
at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:35)
at io.quarkus.opentelemetry.runtime.AutoConfiguredOpenTelemetrySdkBuilderCustomizer_TracerProviderCustomizer_Bean.get(Unknown Source)
at io.quarkus.opentelemetry.runtime.AutoConfiguredOpenTelemetrySdkBuilderCustomizer_TracerProviderCustomizer_Bean.get(Unknown Source)
at io.quarkus.arc.impl.InstanceImpl.getBeanInstance(InstanceImpl.java:325)
at io.quarkus.arc.impl.InstanceImpl$InstanceIterator.next(InstanceImpl.java:363)
at io.quarkus.opentelemetry.runtime.OpenTelemetryRecorder$1.apply(OpenTelemetryRecorder.java:74)
at io.quarkus.opentelemetry.runtime.OpenTelemetryRecorder$1.apply(OpenTelemetryRecorder.java:51)
at io.opentelemetry.api.OpenTelemetry_Le6zQbzkojAYO_OiKIQWJf4lGa4_Synthetic_Bean.createSynthetic(Unknown Source)
... 30 more
Caused by: java.lang.IllegalStateException: Unable to install OTLP Exporter
at io.quarkus.opentelemetry.runtime.exporter.otlp.OTelExporterRecorder$1.apply(OTelExporterRecorder.java:75)
at io.quarkus.opentelemetry.runtime.exporter.otlp.OTelExporterRecorder$1.apply(OTelExporterRecorder.java:48)
at io.quarkus.opentelemetry.runtime.exporter.otlp.LateBoundBatchSpanProcessor_shT2wDm9BfI89PAXjVYmrqS0z6k_Synthetic_Bean.createSynthetic(Unknown Source)
... 63 more
Caused by: java.lang.IllegalArgumentException: Unsupported OTLP protocol http specified. Please check `quarkus.otel.exporter.otlp.traces.protocol` property
at io.quarkus.opentelemetry.runtime.exporter.otlp.OTelExporterRecorder$1.createSpanExporter(OTelExporterRecorder.java:94)
at io.quarkus.opentelemetry.runtime.exporter.otlp.OTelExporterRecorder$1.apply(OTelExporterRecorder.java:63)
... 65 more |
When setting the protocol to http/protobuf the exception disappears. The documentation was not clear about it.
Maybe that should be adjusted. Now that it is set up, sending traces over http/protobuf results in the following error: Which still leads to the question whether the integration of Quarkus with http/protobuf Otel exporter like for dynatrace in this case is already confirmed to be mature? |
Adding this so it could help someone.
Closing this issue. |
Correct @mnguetsa. Those paths are never referred in the quarkus documentation here: https://quarkus.io/guides/opentelemetry |
Add hint about exporter collector protocol on generic gRPC error
Relates to: quarkusio#38963 (cherry picked from commit 74f0221)
Describe the bug
When exporting opentelemetry traces to dynatrace using the Quarkus opentelemetry extension, I get the following WARNING in the console logs all the time:
WARNING [io.qua.ope.run.exp.otl.VertxGrpcExporter] (vert.x-eventloop-thread-9) Failed to export spans. Server responded with error message: null
Even though there is not connection/authentication problem. When contacting the endpiont manually I get a HTTP 200 with an empty json response.
why is Quarkus sending the warning and saying it failed to export the spans?
Thanks in advance
Update:
I changed the default exporter protocol from grpc to http to adhere with dynatrace http/protobuf. http/protobuf is supported by Quarkus according to the doc when the exporter is set to http. However I get this exception that the exporter cannot be installed when starting the app:
Expected behavior
No response
Actual behavior
No response
How to Reproduce?
Checkout dynatrace branch of https://github.com/mnguetsa/quarkus-reproducer. Just start the Quarkus application and the error will appear in the console logs saying the exporter cannot be installed.
Output of
uname -a
orver
No response
Output of
java -version
No response
Quarkus version or git rev
3.7.3
Build tool (ie. output of
mvnw --version
orgradlew --version
)No response
Additional information
No response
The text was updated successfully, but these errors were encountered: