Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Distinct operator returning too many log lines #9594

Open
svennergr opened this issue Jun 1, 2023 · 4 comments · May be fixed by #9624
Open

Distinct operator returning too many log lines #9594

svennergr opened this issue Jun 1, 2023 · 4 comments · May be fixed by #9624
Labels
type/bug Somehing is not working as expected

Comments

@svennergr
Copy link
Contributor

svennergr commented Jun 1, 2023 •
edited
Loading

Describe the bug
I was using the new distinct operator which returned more log lines than expected.

To Reproduce
Steps to reproduce the behavior:

  1. Started Loki
{"version":"2.8.2","revision":"9f809eda7","branch":"HEAD","buildUser":"root@309137d15b61","buildDate":"2023-05-03T11:09:58Z","goVersion":""}
  1. Query: {place="moon"} | json | distinct level | line_format "{{.level}}"
  2. Results where multiple loglines with the same level.

Expected behavior
I expected to see distinct levels.

Environment:

  • Loki:
{"version":"2.8.2","revision":"9f809eda7","branch":"HEAD","buildUser":"root@309137d15b61","buildDate":"2023-05-03T11:09:58Z","goVersion":""}

Screenshots, Promtail config, or terminal output

image
@svennergr svennergr added the type/bug Somehing is not working as expected label Jun 1, 2023
@svennergr
Copy link
Contributor Author

fyi @liguozhong

@liguozhong
Copy link
Contributor

liguozhong commented Jun 1, 2023 •
edited
Loading

Thank you for your feedback.
I will take a look at this bug in the next two weeks.

@liguozhong
Copy link
Contributor

I can reproduce this bug. I will try to fix it.
image

@liguozhong liguozhong linked a pull request Jun 5, 2023 that will close this issue
Open
6 tasks
@liguozhong
Copy link
Contributor

#9624
fixed in this pr.

@kavirajk kavirajk mentioned this issue Aug 25, 2023
Merged
7 tasks
kavirajk added a commit that referenced this issue Aug 25, 2023
@kavirajk
Revert "LogQL: Introduce distinct (#8662)" (#10356)
c030217 
**What this PR does / why we need it**:
This reverts commit 38b298c.

We are removing this PR #8662 because [known issue tracked
here](#9594) and some issues we
facing internally with query splitting and sharding work(hard to
parallelize, merging two `distinct` sets can have duplicates, hard to
make it work with metric queries, etc).

We decided best to remove it before make it to public release.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)

---------

Signed-off-by: Kaviraj <[email protected]>
@grafanabot grafanabot mentioned this issue Aug 25, 2023
Merged
7 tasks
grafanabot pushed a commit that referenced this issue Aug 25, 2023
@kavirajk @grafana-delivery-bot
Revert "LogQL: Introduce distinct (#8662)" (#10356)
c5a078c 
**What this PR does / why we need it**:
This reverts commit 38b298c.

We are removing this PR #8662 because [known issue tracked
here](#9594) and some issues we
facing internally with query splitting and sharding work(hard to
parallelize, merging two `distinct` sets can have duplicates, hard to
make it work with metric queries, etc).

We decided best to remove it before make it to public release.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)

---------

Signed-off-by: Kaviraj <[email protected]>
(cherry picked from commit c030217)
kavirajk added a commit that referenced this issue Aug 25, 2023
@grafanabot @kavirajk
[release-2.9.x] Revert "LogQL: Introduce distinct (#8662)" (#10358)
6bced83 
Backport c030217 from #10356

---

**What this PR does / why we need it**:
This reverts commit 38b298c.

We are removing this PR #8662 because [known issue tracked
here](#9594) and some issues we
facing internally with query splitting and sharding work(hard to
parallelize, merging two `distinct` sets can have duplicates, hard to
make it work with metric queries, etc).

We decided best to remove it before make it to public release.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)

Co-authored-by: Kaviraj Kanagaraj <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug Somehing is not working as expected
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[bugfix] logq: fix Distinct operator returning too many log lines liguozhong/loki
2 participants
@svennergr @liguozhong