[AIP-49] Decouple metrics clients and validators into their own modules #30802
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Interesting. Looking into it.
Suggestions always welcome |
ferruzzi
force-pushed
the
ferruzzi/otel/m3-decoupling
branch
from
0d1d3f2 to
fbf5e93
Compare
potiuk
approved these changes
Apr 23, 2023
|
Needs conflict resolving |
ferruzzi
force-pushed
the
ferruzzi/otel/m3-decoupling
branch
from
fbf5e93 to
6f40108
Compare
o-nikolas
approved these changes
Apr 24, 2023
|
lgtm, I like the refactor, very useful to break down these massive modules a bit. I've been doing similar stuff in the CLI space. |
|
Agree. |
ferruzzi
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Part of the ongoing OTel implementation. Should not contain any logic changes, just moving things around into a more modular "plugin" style layout like we have elsewhere instead of the "monolithic" stats.py we currently have.
@howardyoo and @potiuk: This is (my proposal for) the change we discussed yesterday.
Motivation
airflow/stats.pycurrently contains the metrics interface as well as implementations of the interface for StatsD and Datadog, some protocols, and validators (along with their interface). This PR does not change any logic, simply moves related content into relevant modules. For example All StatsD-specific code now lives inairflow/metrics/statsd-logger.py. This is to make the stats.py file a bit cleaner, make adding OTel easier, and possibly make removing the StatsD easier later if we choose to go that route.When I add the OpenTelemetry code, it will live in
airflow/metrics/otel-logger.pyand simply import and implement the interfaces fromstats.pyandmetrics/protocols.pyinstead of adding it to the existing growing monolith instats.py.Testing
Passes
breeze static-checks --all-filesandbreeze testing tests --run-in-parallellocally.cc: @o-nikolas @vincbeck @vandonr-amz @syedahsn