Redriven Step Functions Trace Merging #598

avedmala · 2024-12-04T15:33:06Z

What does this PR do?

Adds support for Step Functions trace merging in Redrive cases

We previously used hash(ExecutionId # StateName # StateEnteredTime) for spanID calculation but these values are identical across redrives for a Lambda task state. The new approach also adds a RedriveCount to the end of the hash but omits this value when it is 0 to have easy backwards compatability.

Motivation

https://github.com/DataDog/logs-backend/pull/84188

Testing Guidelines

Redriven Trace with Lambda and Trace Merging

Types of Changes

Bug fix
New feature
Breaking change
Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

This PR's description is comprehensive
This PR contains breaking changes that are documented in the description
This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
This PR impacts documentation, and it has been updated (or a ticket has been logged)
This PR's changes are covered by the automated tests
This PR collects user input/sensitive content into Datadog
This PR passes the integration tests (ask a Datadog member to run the tests)

kimi-p · 2024-12-16T15:06:16Z

src/trace/context/extractors/step-function.spec.ts

+      expect(traceContext).not.toBeNull();
+
+      expect(traceContext?.toTraceId()).toBe("1139193989631387307");
+      expect(traceContext?.toSpanId()).toBe("3615526381891637131");


Could you link the PR line in the reducer that matches this test?

Also, is the plan to match traceID only but not spanID?

Here's the matching line from logs-backend

Yes the lambda span should be in the same trace but we want the spanID to be different from the original when redrive count was 0

Can you link the test just to verify that the two matches? Thanks!

kimi-p

LGTM

avedmala · 2024-12-17T17:11:56Z

/merge

dd-devflow · 2024-12-17T17:12:15Z

Devflow running: `/merge`

View all feedbacks in Devflow UI.

2024-12-17 17:12:15 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in main is 0s.

2024-12-17 17:21:39 UTC 🚨 MergeQueue: This merge request is in error

mergequeue build completed successfully, but the github api returned an error while merging the pr.
It's probably because:

target branch of PR is restricted to only allow up-to-date branches, but the pr is now outdated

Details

Error: PUT https://api.github.com/repos/DataDog/datadog-lambda-js/pulls/598/merge: 405 15 of 15 required status checks are expected. []

FullStacktrace:
activity error (type: github.GithubService_MergePullRequest, scheduledEventID: 41, startedEventID: 42, identity: 1@github-worker-66d47658fd-jwq4x@): PUT https://api.github.com/repos/DataDog/datadog-lambda-js/pulls/598/merge: 405 15 of 15 required status checks are expected. [] (type: GitFailure, retryable: false): PUT https://api.github.com/repos/DataDog/datadog-lambda-js/pulls/598/merge: 405 15 of 15 required status checks are expected. [] (type: ErrorResponse, retryable: true)

avedmala · 2024-12-17T18:00:08Z

/merge

dd-devflow · 2024-12-17T18:00:19Z

Devflow running: `/merge`

View all feedbacks in Devflow UI.

2024-12-17 18:00:19 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-12-17 18:02:25 UTC ℹ️ MergeQueue: merge request added to the queue

The median merge time in main is 0s.

2024-12-17 18:09:19 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 2fae7e7:

integration test (node18)

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

Investigate the failures and when ready, re-add your pull request to the queue!
Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

avedmala · 2024-12-17T21:04:03Z

/merge

dd-devflow · 2024-12-17T21:04:12Z

Devflow running: `/merge`

View all feedbacks in Devflow UI.

2024-12-17 21:04:12 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in main is 0s.

2024-12-17 21:12:59 UTC ℹ️ MergeQueue: This merge request was merged

avedmala added 4 commits December 4, 2024 10:32

use redrive count to generate step functions parent ID

3a86242

dont use redrive_count when 0

c60e8f0

fix tests

ea62b9b

fix redrive count type

20e020f

avedmala marked this pull request as ready for review December 16, 2024 14:26

avedmala requested a review from a team as a code owner December 16, 2024 14:26

avedmala changed the title ~~use redrive count to generate step functions parent ID~~ Redriven Step Functions Trace Merging Dec 16, 2024

kimi-p reviewed Dec 16, 2024

View reviewed changes

kimi-p approved these changes Dec 16, 2024

View reviewed changes

added link to matching snapshot in logs-backend

eda6b92

avedmala mentioned this pull request Dec 17, 2024

Redriven Step Functions Trace Merging DataDog/datadog-lambda-python#545

Merged

11 tasks

dd-devflow bot added mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: error and removed mergequeue-status: queued mergequeue-status: in_progress labels Dec 17, 2024

Merge branch 'main' into avedmala/sfn-redrive-merging

7acae02

dd-devflow bot added mergequeue-status: waiting mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected and removed mergequeue-status: error mergequeue-status: waiting mergequeue-status: queued mergequeue-status: in_progress labels Dec 17, 2024

dd-devflow bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: rejected mergequeue-status: queued labels Dec 17, 2024

dd-mergequeue bot merged commit 1a75734 into main Dec 17, 2024
32 checks passed

dd-devflow bot removed the mergequeue-status: in_progress label Dec 17, 2024

dd-mergequeue bot deleted the avedmala/sfn-redrive-merging branch December 17, 2024 21:12

dd-devflow bot added the mergequeue-status: done label Dec 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redriven Step Functions Trace Merging #598

Redriven Step Functions Trace Merging #598

avedmala commented Dec 4, 2024 •

edited

Loading

kimi-p Dec 16, 2024

avedmala Dec 16, 2024

kimi-p Dec 16, 2024

kimi-p left a comment

avedmala commented Dec 17, 2024

dd-devflow bot commented Dec 17, 2024 •

edited

Loading

avedmala commented Dec 17, 2024

dd-devflow bot commented Dec 17, 2024 •

edited

Loading

avedmala commented Dec 17, 2024

dd-devflow bot commented Dec 17, 2024 •

edited

Loading

Redriven Step Functions Trace Merging #598

Redriven Step Functions Trace Merging #598

Conversation

avedmala commented Dec 4, 2024 • edited Loading

What does this PR do?

Motivation

Testing Guidelines

Types of Changes

Check all that apply

kimi-p Dec 16, 2024

Choose a reason for hiding this comment

avedmala Dec 16, 2024

Choose a reason for hiding this comment

kimi-p Dec 16, 2024

Choose a reason for hiding this comment

kimi-p left a comment

Choose a reason for hiding this comment

avedmala commented Dec 17, 2024

dd-devflow bot commented Dec 17, 2024 • edited Loading

Devflow running: /merge

avedmala commented Dec 17, 2024

dd-devflow bot commented Dec 17, 2024 • edited Loading

Devflow running: /merge

What to do next?

avedmala commented Dec 17, 2024

dd-devflow bot commented Dec 17, 2024 • edited Loading

Devflow running: /merge

avedmala commented Dec 4, 2024 •

edited

Loading

dd-devflow bot commented Dec 17, 2024 •

edited

Loading

Devflow running: `/merge`

dd-devflow bot commented Dec 17, 2024 •

edited

Loading

Devflow running: `/merge`

dd-devflow bot commented Dec 17, 2024 •

edited

Loading

Devflow running: `/merge`