Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[w3c] Support W3C trace context propagation #429

Merged

Conversation

duncanista
Copy link
Contributor

@duncanista duncanista commented Nov 7, 2023

What does this PR do?

TLDR; Refactor code and delegate trace extraction to the tracer.

This PR does a couple things:

  1. Refactoring functions into classes to make trace context extraction simpler.
  2. Delegate the trace context extraction to the tracer, instead of doing it manually only for Datadog headers.
  3. Support W3C and Datadog headers extraction – actually, any supported by the current tracer.
  4. Add multiple new unit tests for every component.
  5. Separates Xray and Step Functions into services.

Motivation

Our library was extracting tracing context only for Datadog headers, the need to extract from W3C trace context is expected and already being done by the dd-trace package. Therefore, we should delegate our previous manual extraction to the already predefined functions.

Testing Guidelines

  • Unit tests
  • Integration tests
  • Manual tests

Services using W3C context in headers in AWS Lambda.

Screenshot 2023-11-07 at 8 26 07 PM

Screenshot 2023-11-07 at 8 36 32 PM

Additional Notes

  • Patching for HTTP and console when no tracer is available will fallback to always propagate Datadog Headers only.
  • Soft deprecation of traceID and parentID in the custom extractor expected value is in place, in the next breaking change, this will be removed in favor of traceId and parentId.

Types of Changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)

aiming to enhance its use, created for xray trace context handling, now we expect it to use it for all of the tracing context
simple wrapper to use instead of `TraceContext`, this allows us to type the object extracted by the tracer. I decided to wrap it into a class, to easily managea black-box class – yes, I know we are importing it in one of the constructors, but that is the easiest way to create one for trace context with Xray, StepFunction event, and for Custom Extractors, so we can tell the tracer what is a child of what.
make it complaint to use the refactored classes, also updated its unit tests to mock the new methods being used
the main point of this class is to create a singleton state manager for the `StepFunctionContext` interface, it will run per invocation, and should not interfere with new invocations
this new class handles all logic related to Xray, previously, it was scattered in `context.ts`, but it actually didnt make sense, since we had multiple functions which should be in the same place, we could also keep a state since we are relying on a header and context which will always be deterministic, so no need to calculate it again in multiple functions
this file will be in charge of extracting the trace context from the incoming context, event, step function context, or xray, separating the logic into an adapter pattern for a much simpler and easier way of handling this extraction
refactoring code to make sure we correctly handling an HTTP event for trace context
allow custom extractor to be in a separate class, which then uses a helper method to generate an `SpanContextWrapper` to be returned, had to deprecate some fields, so to avoid breaking changes, I have marked some fields as nullable
basically relies on `http.ts` to work
same as other extractors, separation of concerns into its own file
same as other extractors, separation of concerns into its own file
same as other extractors, separation of concerns into its own file
same as other extractors, separation of concerns into its own file, I think we could do some more refactoring, but for now, I will leave sns-sqs and eventbridge-sqs as separate files
same as other extractors, separation of concerns into its own file
same as other extractors, separation of concerns into its own file, related to sns-sqs comment, maybe in the future we could explore refactoring this extraction, but for now, will stay as is
same as other extractors, separation of concerns into its own file, also made sure legacy payloads still work
same as other extractors, separation of concerns into its own file, uses the singleton class to get context
basically to export them from the root
updated unit tests to make sure everything works as before with the new methods and mockings
w3c will not be supported here, since we would have to do manual extraction and injection, this will not be supported for customers not using `dd-trace`, so the refactoring is just to use the new methods and mockings
w3c will not be supported here, since we would have to do manual extraction and injection, this will not be supported for customers not using `dd-trace`, so the refactoring is just to use the new methods and mockings, if a new user wants to correlate traces with logs without `dd-trace`, they will have to use the datadog headers, this might be up for removal in the future
make use of the new services, also refactored unit tests to work as expected
might need to re-explore this method, but for safety, I am adding it just to make sure we are indeed creating a new instance of the service on every invocation
export from the right file
this class aims to replace `event-type-guards.ts`, I think the file name is confusing, and we should use a class as a namespace here, nonetheless, I wonder if this change is necessary at all, since it would mostly use the same exact logic, but allowing it to be exported as a single class with static methods, this is exactly the same as what we do when importing the whole file.
decomissioning a large file and its unit tests, made sure to refactor every single method into multiple files and classes
@codecov-commenter
Copy link

codecov-commenter commented Nov 7, 2023

Codecov Report

Attention: 66 lines in your changes are missing coverage. Please review.

Comparison is base (109a606) 81.19% compared to head (0e25e8c) 81.96%.

❗ Current head 0e25e8c differs from pull request most recent head 1871ba7. Consider uploading reports for the commit 1871ba7 to get more accurate results

Files Patch % Lines
src/trace/span-context-wrapper.ts 56.25% 13 Missing and 1 partial ⚠️
src/trace/xray-service.ts 91.26% 8 Missing and 1 partial ⚠️
src/trace/step-function-service.ts 91.86% 6 Missing and 1 partial ⚠️
src/trace/context/extractor.ts 90.56% 5 Missing ⚠️
src/trace/context/extractors/sqs.ts 58.33% 4 Missing and 1 partial ⚠️
src/trace/context/extractors/event-bridge.ts 66.66% 3 Missing and 1 partial ⚠️
src/trace/context/extractors/http.ts 91.83% 3 Missing and 1 partial ⚠️
src/trace/context/extractors/sns.ts 76.47% 3 Missing and 1 partial ⚠️
src/trace/context/extractors/event-bridge-sqs.ts 80.00% 1 Missing and 2 partials ⚠️
src/trace/context/extractors/kinesis.ts 81.25% 1 Missing and 2 partials ⚠️
... and 5 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #429      +/-   ##
==========================================
+ Coverage   81.19%   81.96%   +0.76%     
==========================================
  Files          50       54       +4     
  Lines        2042     2179     +137     
  Branches      467      506      +39     
==========================================
+ Hits         1658     1786     +128     
- Misses        323      330       +7     
- Partials       61       63       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@duncanista
Copy link
Contributor Author

duncanista commented Nov 7, 2023

It's normal for integration tests to not pass, since now, console patching only happens when dd-trace is available, which is a contradiction – and a breaking change. Will need to fix it either by doing a try-catch, but this might require to handle scenarios where we can get trace-context when there's no tracer: Xray, StepFunctionContext, and Custom extractor.

-- edit: fixed

…erate the context everytime, I tried to remove this by having the `header` and `context` as properties, but somehow, it ends up being `undefined`, therefore the switch back
@duncanista duncanista changed the base branch from main to jordan.gonzalez/SVLS-3939/re-organize-trace-context-extraction November 8, 2023 21:47
Copy link
Contributor

@astuyve astuyve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple quick notes

@duncanista duncanista changed the base branch from jordan.gonzalez/SVLS-3939/re-organize-trace-context-extraction to main November 8, 2023 22:56
@duncanista duncanista marked this pull request as ready for review November 9, 2023 02:58
@duncanista duncanista requested a review from a team as a code owner November 9, 2023 02:58
@astuyve astuyve dismissed their stale review November 14, 2023 20:41

I pushed commits to fix it

@astuyve astuyve requested a review from purple4reina November 20, 2023 12:23
@astuyve astuyve merged commit f05c600 into main Nov 20, 2023
@astuyve astuyve deleted the jordan.gonzalez/SVLS-3939/support-w3c-trace-context-headers branch November 20, 2023 17:37
@samchungy samchungy mentioned this pull request Nov 26, 2023
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants