Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow using prometheus-client crate with PrometheusClientLayer #3134

Merged
merged 18 commits into from
Sep 19, 2023

Conversation

flaneur2020
Copy link
Contributor

@flaneur2020 flaneur2020 commented Sep 19, 2023

I'm currently working on switching to prometheus from metrics-rs in databend at databendlabs/databend#12787 , but unfortunately the PrometheusLayer in opendal uses the tikv/prometheus-rs crate, but databend uses prometheus-client from the prometheus official.

This PR addes a feature option use-prometheus-client to allow switching to prometheus-client with the PrometheusLayer. When use-prometheus-client is enabled, we can pass a prometheus_client::registry::Registry to a PrometheusLayer.

To support both of the prometheus client, this PR also make a small refactor to collect the metrics writing logics into a trait PrometheusLayerMetrics.

I believe the official prometheus may have longer term of support, maybe we can have a schedule to migrate to it while deprecate the older prometheus library's support?

@flaneur2020 flaneur2020 marked this pull request as ready for review September 19, 2023 07:13
@Xuanwo
Copy link
Member

Xuanwo commented Sep 19, 2023

I would add a new layer called PrometheusClientLayer instead of maintaining two different features. Playing with features is not fun 😢

@flaneur2020
Copy link
Contributor Author

@Xuanwo you're right, what a pain. I've refactored to another PrometheusClientLayer, PTAL

@flaneur2020 flaneur2020 changed the title feat: allow using prometheus-client crate with PrometheusLayer feat: allow using prometheus-client crate with PrometheusClientLayer Sep 19, 2023
core/src/layers/prometheus_client.rs Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus_client.rs Show resolved Hide resolved
@flaneur2020 flaneur2020 requested a review from Xuanwo September 19, 2023 11:11
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, let's go!

@Xuanwo
Copy link
Member

Xuanwo commented Sep 19, 2023

The only thing left is to make our CI happy.

@Xuanwo Xuanwo merged commit 30de95e into apache:main Sep 19, 2023
Young-Flash pushed a commit to Young-Flash/opendal that referenced this pull request Sep 19, 2023
apache#3134)

* refactor prometheus layer

* add prometheus-client to deps

* chore: simplify imports

* refactor the metrics into a trait

* feat: add implementation with prometheus-client

* fix: allow using different trait

* cargo fmt

* refactor: add a seperate layer

* fix: docs

* fix typo

* fix: cargo fmt

* add a prefix

* use structed labels

* use labels in array

* remove the unused metrics

* rename stats to metrics

* record request duration in wrapper

* fix fmt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants