Metrics instrumentation celery

[srikanthccv]

Relevant semconv: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/semantic_conventions/instrumentation/kafka.md

[sanketmehta28]

Hi Can you assign it to me?

[TheAnshul756]

Hello @sanketmehta28
If you are stuck or have not started working on it yet, can you please assign this to me?
CC @srikanthccv

[levivm]

Hello @srikanthccv @TheAnshul756 this means that I can't use Opentelemetry Metrics under Celery ? Right now I'm trying to use it but some metrics are getting lost, I assume it's because the normal exporter doesn't work correctly in celery/worker-based projects, right ?

[srikanthccv]

@levivm I didn't understand your question. This issue is for adding metrics instrumentation for opentelemetry-instrumentation-celery. Today it only supports tracing.

[levivm]

@srikanthccv I want to send metrics not span from a celery worker using OpenTelemetry, but it seems some metrics are getting lost, so, I don't know if it is because OpenTelemetry Metrics is not yet supported to be used along celery ? Thanks in advance for you help.

[srikanthccv]

The celery instrumentation doesn't have metrics instrumentation yet and this is a tracker issue for it.

[TheAnshul756]

Hello @srikanthccv
I wanted to know what metric semantic convention to follow for task queues like Celery?

[srikanthccv]

We will do something similar to this https://flower.readthedocs.io/en/latest/prometheus-integration.html#available-metrics. Since this framework is specific to Python projects it won't probably be added to spec but we can provide these metrics using OTEL Metrics API.

[Akochavi]

Hi, can I work on it or you already work on it? @TheAnshul756

[TheAnshul756]

Sure you can work on it.

[Akochavi]

@srikanthccv Hi, I try to add Gauge metrics and I see that there isn't record option for Gauge (For example when I use Histogram, the option is exist).
Do you know if this option exists in some form?
Is there a situation where they didn't implement it until they needed to? Or maybe you don't need to call anything at all and it should work that way?
Because no one has used it before so there is no reference at all.

[srikanthccv]

There is no sync gauge in metric API/SDK specification. However, there is an async gauge. What gauge metrics did you have to add?

[Akochavi]

@srikanthccv

• Number of tasks currently executing in this worker
• Number of tasks of given type prefetched at a worker

[srikanthccv]

You can use UpDownCounter?

[Akochavi]

I use a histogram, I found it more suitable

[Akochavi]

@srikanthccv Why we don't merge it?
I see that I got approve.

[srikanthccv]

There are unresolved conversations. It will be merged when they are addressed.

[Akochavi]

@srikanthccv All the conversations resolved.