Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query-frontend: Improve started call context #4768

Closed
aymericDD opened this issue Oct 12, 2021 · 2 comments
Closed

query-frontend: Improve started call context #4768

aymericDD opened this issue Oct 12, 2021 · 2 comments

Comments

@aymericDD
Copy link
Contributor

aymericDD commented Oct 12, 2021

Is your proposal related to a problem?

When the --log.request.decision is set to LogStartAndFinishCall with a --log.level=debug you have this kind of stdout :

Logs

ts=2021-10-12T07:31:54.28451398Z caller=level.go:63 protocol=http http.component=server level=debug http.start_time="2021-10-12 07:31:54.28449829 +0000 UTC m=+84539.248499213" msg="started call"
ts=2021-10-12T07:31:54.289747293Z caller=level.go:63 protocol=http http.component=server level=debug http.start_time="2021-10-12 07:31:54.289738502 +0000 UTC m=+84539.253739412" msg="started call"
ts=2021-10-12T07:31:54.310156358Z caller=level.go:63 protocol=http http.component=server http.method="GET /api/v1/query_range?query=xxxxx" http.request_id=01FHSPMESCBPDA5N4EQCFA9807 http.status_code=200 http.time_ms=25.611 http.remote_addr=10.10.235.6:7533 thanos.method_name=query-frontend level=debug msg="finished call"
ts=2021-10-12T07:31:54.31237475Z caller=level.go:63 protocol=http http.component=server http.method="GET /api/v1/query?query=xxxxx" http.request_id=01FHSPMESHE03ZQ2S7DVW5YQZ7 http.status_code=200 http.time_ms=22.615 http.remote_addr=10.10.235.11:5139 thanos.method_name=query-frontend level=debug msg="finished call"

  • The first problem here is you cannot know witch start call is attached to its finished call.
  • The second problem is you don't know witch request is executed at the start call. Imagine the pod is killed during the execution of the request you cannot know witch request caused the outage.

Describe the solution you'd like

I proposed to add more context to the start call. The http.method the http.requestId and the thanos.method_name.

Example:

Logs

ts=2021-10-12T07:46:21.853548404Z caller=level.go:63 protocol=http http.component=server level=debug http.start_time="2021-10-12 09:46:21.853534365 +0200 CEST m=+14.444830295" http.method="GET /api/v1/query?query=xxx" http.request_id=01FHSQEY0XM8G0DN06B9XM61E7 thanos.method_name=query-frontend msg="started call"
ts=2021-10-12T07:46:21.904385711Z caller=level.go:63 protocol=http http.component=server http.method="GET https://a01monithor-query.cdweb.biz/api/v1/query?query=xxxxx" http.request_id=01FHSQEY0XM8G0DN06B9XM61E7 http.status_code=200 http.time_ms=50.844 http.remote_addr=127.0.0.1:36476 thanos.method_name=query-frontend level=debug msg="finished call"

Describe alternatives you've considered

You can use a revers proxy and use its logs.

Additional context

RAS

@aymericDD aymericDD changed the title Improve started call context query-frontend: Improve started call context Oct 12, 2021
@matej-g
Copy link
Collaborator

matej-g commented Oct 12, 2021

Agreed, this would definitely be an improvement! 👍

@yeya24
Copy link
Contributor

yeya24 commented Nov 9, 2021

Close in favor of #4769

@yeya24 yeya24 closed this as completed Nov 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants