Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.15](backport #40451) Add a tracer to send (legacy) Azure Event Hub SDK logs to logp #40603

Merged
merged 2 commits into from
Aug 23, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Aug 23, 2024

Proposed commit message

Add a tracer for the github.com/devigned/tab package to route logs from the legacy event hub SDK to the logp package.

The legacy event hub SDK uses the github.com/devigned/tab package to handle traces and logs.

However, this package is old and seems unmaintained (last commit: 3 years ago). Since it only supports OpenTracing or OpenCensus, in Beats, the library defaults to a NoOpTracer, so we don’t see anything happening inside the legacy processor.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

I don't expect the trace to impact users or other Beats components. It seems the legacy event hub SDK is the only dependency using this old package:

$ go mod why -m github.com/devigned/tab

# github.com/devigned/tab
github.com/elastic/beats/v7/x-pack/filebeat/input/azureeventhub
github.com/Azure/azure-event-hubs-go/v3
github.com/devigned/tab

Related issues

Logs

Start Filebeat using a valid azure-eventhub config, and you will be able to read the log messages from the processor and the leaser/checkpointer components:

$ pbpaste | grep '^{' | jq -r 'select(."log.origin"."file.name" == "azureeventhub/tracer.go") | [."@timestamp",."log.level",.message] | @tsv'

2024-08-07T16:17:12.305+0200	debug	successful rpc on link 494e91ef-8e06-4460-b89b-361d44c64f75: status code 200 and description:
2024-08-07T16:17:17.306+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb": running scan
2024-08-07T16:17:19.667+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb": acquired: [{"partitionID":"1","epoch":21,"owner":"86edc94f-dd34-44d3-8a43-ecf3cf4c77fb","checkpoint":{"offset":"201863710488","sequenceNumber":8568,"enqueueTime":"2024-08-07T13:35:51.837Z"},"state":"available","token":"32ab10ca-cc08-42a8-83fd-1dc436f25abc"} {"partitionID":"0","epoch":23,"owner":"86edc94f-dd34-44d3-8a43-ecf3cf4c77fb","checkpoint":{"offset":"206158444904","sequenceNumber":8565,"enqueueTime":"2024-08-07T13:26:15.225Z"},"state":"available","token":"e4aa4ecd-ab0f-4984-92cd-a4c6dbcfa433"}], not acquired: []
2024-08-07T16:17:19.667+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "1", epoch 21: running...
2024-08-07T16:17:19.667+0200	debug	creating a new receiver
2024-08-07T16:17:21.349+0200	debug	negotiating claim for audience amqps://mbranca-general.servicebus.windows.net/activitylogs/ConsumerGroups/$Default/Partitions/1 with token type servicebus.windows.net:sastoken and expiry of 1723047441
2024-08-07T16:17:21.562+0200	debug	successful rpc on link 3f9e7e50-f012-4692-8172-8b8d5c9dde17: status code 202 and description: Accepted
2024-08-07T16:17:21.563+0200	debug	negotiated with response code 202 and message: Accepted
2024-08-07T16:17:22.293+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "0", epoch 23: running...
2024-08-07T16:17:22.293+0200	debug	creating a new receiver
2024-08-07T16:17:24.705+0200	debug	negotiating claim for audience amqps://mbranca-general.servicebus.windows.net/activitylogs/ConsumerGroups/$Default/Partitions/0 with token type servicebus.windows.net:sastoken and expiry of 1723047445
2024-08-07T16:17:25.013+0200	debug	successful rpc on link 6bb4086b-b8ae-46ca-87dc-ef4e18c15db2: status code 202 and description: Accepted
2024-08-07T16:17:25.013+0200	debug	negotiated with response code 202 and message: Accepted
2024-08-07T16:17:30.261+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "1", epoch 21: lease renewed
2024-08-07T16:17:32.620+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "0", epoch 23: lease renewed
2024-08-07T16:17:35.807+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb": running scan
2024-08-07T16:17:36.566+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb": acquired: [], not acquired: [{"partitionID":"0","epoch":23,"owner":"86edc94f-dd34-44d3-8a43-ecf3cf4c77fb","checkpoint":{"offset":"206158444904","sequenceNumber":8565,"enqueueTime":"2024-08-07T13:26:15.225Z"},"state":"leased","token":"e4aa4ecd-ab0f-4984-92cd-a4c6dbcfa433"} {"partitionID":"1","epoch":21,"owner":"86edc94f-dd34-44d3-8a43-ecf3cf4c77fb","checkpoint":{"offset":"201863710488","sequenceNumber":8568,"enqueueTime":"2024-08-07T13:35:51.837Z"},"state":"leased","token":"32ab10ca-cc08-42a8-83fd-1dc436f25abc"}]
2024-08-07T16:17:43.233+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "0", epoch 23: lease renewed
2024-08-07T16:17:46.407+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb": running scan
2024-08-07T16:17:47.382+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb": acquired: [], not acquired: [{"partitionID":"0","epoch":23,"owner":"86edc94f-dd34-44d3-8a43-ecf3cf4c77fb","checkpoint":{"offset":"206158444904","sequenceNumber":8565,"enqueueTime":"2024-08-07T13:26:15.225Z"},"state":"leased","token":"e4aa4ecd-ab0f-4984-92cd-a4c6dbcfa433"} {"partitionID":"1","epoch":21,"owner":"86edc94f-dd34-44d3-8a43-ecf3cf4c77fb","checkpoint":{"offset":"201863710488","sequenceNumber":8568,"enqueueTime":"2024-08-07T13:35:51.837Z"},"state":"leased","token":"32ab10ca-cc08-42a8-83fd-1dc436f25abc"}]
2024-08-07T16:17:50.100+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "1", epoch 21: lease renewed
2024-08-07T16:17:52.965+0200	debug	eph "86edc94f-dd34-44d3-8a43-ecf3cf4c77fb", partition "0", epoch 23: lease renewed
```<hr>This is an automatic backport of pull request #40451 done by [Mergify](https://mergify.com).

Add a tracer for the github.com/devigned/tab package to route logs from the legacy event hub SDK to the logp package.

The legacy event hub SDK uses the [github.com/devigned/tab](http://github.com/devigned/tab) package to handle traces and logs.

However, this package is old and seems unmaintained (last commit: 3 years ago). Since it only supports OpenTracing or OpenCensus, in Beats, the library defaults to a `NoOpTracer`, so we don’t see anything happening inside the legacy processor.

(cherry picked from commit e64ea44)

# Conflicts:
#	NOTICE.txt
@mergify mergify bot requested review from a team as code owners August 23, 2024 13:39
@mergify mergify bot added the backport label Aug 23, 2024
@mergify mergify bot requested a review from a team as a code owner August 23, 2024 13:39
@mergify mergify bot added the conflicts There is a conflict in the backported pull request label Aug 23, 2024
@mergify mergify bot requested review from belimawr and mauri870 and removed request for a team August 23, 2024 13:39
@mergify mergify bot assigned zmoog Aug 23, 2024
Copy link
Contributor Author

mergify bot commented Aug 23, 2024

Cherry-pick of e64ea44 has failed:

On branch mergify/bp/8.15/pr-40451
Your branch is up to date with 'origin/8.15'.

You are currently cherry-picking commit e64ea44140.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   go.mod
	modified:   x-pack/filebeat/include/list.go
	new file:   x-pack/filebeat/input/azureeventhub/tracer.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   NOTICE.txt

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2024
@botelastic
Copy link

botelastic bot commented Aug 23, 2024

This pull request doesn't have a Team:<team> label.

@zmoog zmoog merged commit bdaf6dc into 8.15 Aug 23, 2024
122 checks passed
@zmoog zmoog deleted the mergify/bp/8.15/pr-40451 branch August 23, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport conflicts There is a conflict in the backported pull request needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant