Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/loadbalancingexporter] - Static resolver dropping the data when one of the host is not healthy #32045

Closed
ramanjaneyagupta opened this issue Mar 29, 2024 · 2 comments
Labels
bug Something isn't working exporter/loadbalancing needs triage New item requiring triage

Comments

@ramanjaneyagupta
Copy link

ramanjaneyagupta commented Mar 29, 2024

Component(s)

exporter/loadbalancing

Describe the issue you're reporting

Collector Version: v0.96.0
Configuration:

exporters:
  loadbalancing:
    routing_key: "traceID"
    protocol:
      otlp:
        timeout: 1s
    resolver:
      static:
        hostnames:
        - backend-1:4317
        - backend-2:4317
        - backend-3:4317
        - backend-4:4317 
service:
  pipelines:
    traces:
      receivers:
        - otlp
      exporters:
        - loadbalancing

If one of the host is down in the static list, loadbalancingexporter unable to send the data to healthy host. Instead sending the data to health host it is dropping the data on the floor.

@ramanjaneyagupta ramanjaneyagupta added the needs triage New item requiring triage label Mar 29, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@atoulme atoulme changed the title [exporter/loadbalancingexporter] - Static resolver dropping the data when one of the host is not helathy [exporter/loadbalancingexporter] - Static resolver dropping the data when one of the host is not healthy Mar 30, 2024
@crobert-1 crobert-1 added the bug Something isn't working label Apr 9, 2024
@jpkrohling
Copy link
Member

If one of the host is down in the static list, loadbalancingexporter unable to send the data to healthy host. Instead sending the data to health host it is dropping the data on the floor.

That's the expected behavior, at least for the moment. When using a static list, we expect the hosts to be available to receive the data, even if they are temporarily unavailable (for instance, by using a Kubernetes Statefulset). If your static list isn't that static, the solution is to use a DNS resolver, where the DNS is updated to list only the healthy nodes (like a Kubernetes headless service).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working exporter/loadbalancing needs triage New item requiring triage
Projects
None yet
Development

No branches or pull requests

3 participants