Add a tracer to send (legacy) Azure Event Hub SDK logs to logp #40451
Conversation
The tracer and spanner from github.com/devigned/tab only route logs to the logp package.
botelastic
bot
added
the
needs_team
zmoog
added
enhancement
Team:obs-ds-hosted-services
|
This pull request does not have a backport label.
To fixup this pull request, you need to add the backport labels for the needed
|
|
Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services) |
pierrehilbert
added
the
Team:Elastic-Agent-Data-Plane
|
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane) |
| func (sl logpLogger) Info(msg string, attributes ...tab.Attribute) { | ||
| sl.logger.Info(msg) | ||
| } | ||
|
|
||
| // Error logs a message at error level | ||
| func (sl logpLogger) Error(err error, attributes ...tab.Attribute) { | ||
| sl.logger.Error(err) | ||
| } | ||
|
|
||
| // Fatal logs a message at Fatal level | ||
| func (sl logpLogger) Fatal(msg string, attributes ...tab.Attribute) { | ||
| sl.logger.Fatal(msg) | ||
| } | ||
|
|
||
| // Debug logs a message at Debug level | ||
| func (sl logpLogger) Debug(msg string, attributes ...tab.Attribute) { |
There was a problem hiding this comment.
[Blocker]
Why are the attributes ignored?
you could use the w functions to pass the attributes to logp
There was a problem hiding this comment.
With the current simple and log-focused logsOnlyTracer and logsOnlySpanner implementations, the attributes are nil:
I am targeting 8.15.1 to ship at least the logs because, as of today, we have literally zero visibility into the legacy SDK's inner workings. The logs would be pure gold in a couple of SDHs I am working on.
There was a problem hiding this comment.
I aim to bridge the legacy SDK logs into logp so they will be available in a diagnostics bundle.
I would love to have traces and spans. However, logs alone bring 80% of the value, and they seem achievable in the 8.15.1 FF time frame.
There was a problem hiding this comment.
@AndersonQ, do you think it's okay to ship this logging solution in time for 8.15.1 (this could help in troubleshooting a lot) and invest more in tracing for input v1 and v2 for 8.15.2 and 8.16?
There was a problem hiding this comment.
well, something is better than nothing, even thought I still believe the effort to add the attributes is marginal, I won't block the PR.
There was a problem hiding this comment.
but it'd appreciate a followup PR logging the attributes as well
Add a tracer for the github.com/devigned/tab package to route logs from the legacy event hub SDK to the logp package. The legacy event hub SDK uses the [github.com/devigned/tab](http://github.com/devigned/tab) package to handle traces and logs. However, this package is old and seems unmaintained (last commit: 3 years ago). Since it only supports OpenTracing or OpenCensus, in Beats, the library defaults to a `NoOpTracer`, so we don’t see anything happening inside the legacy processor. (cherry picked from commit e64ea44) # Conflicts: # NOTICE.txt
… SDK logs to logp (#40603) * Add a tracer to send (legacy) Azure Event Hub SDK logs to logp (#40451) Add a tracer for the github.com/devigned/tab package to route logs from the legacy event hub SDK to the logp package. The legacy event hub SDK uses the [github.com/devigned/tab](http://github.com/devigned/tab) package to handle traces and logs. However, this package is old and seems unmaintained (last commit: 3 years ago). Since it only supports OpenTracing or OpenCensus, in Beats, the library defaults to a `NoOpTracer`, so we don’t see anything happening inside the legacy processor. --------- Co-authored-by: Maurizio Branca <[email protected]>
Proposed commit message
Add a tracer for the github.com/devigned/tab package to route logs from the legacy event hub SDK to the logp package.
The legacy event hub SDK uses the github.com/devigned/tab package to handle traces and logs.
However, this package is old and seems unmaintained (last commit: 3 years ago). Since it only supports OpenTracing or OpenCensus, in Beats, the library defaults to a
NoOpTracer, so we don’t see anything happening inside the legacy processor.Checklist
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Disruptive User Impact
I don't expect the trace to impact users or other Beats components. It seems the legacy event hub SDK is the only dependency using this old package:
$ go mod why -m github.com/devigned/tab # github.com/devigned/tab github.com/elastic/beats/v7/x-pack/filebeat/input/azureeventhub github.com/Azure/azure-event-hubs-go/v3 github.com/devigned/tabRelated issues
Logs
Start Filebeat using a valid azure-eventhub config, and you will be able to read the log messages from the processor and the leaser/checkpointer components: