Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dataset quality] Replaced Estimated Data section by doc count in serverless deployments #180582

Closed
flash1293 opened this issue Apr 11, 2024 · 8 comments
Labels
Feature:Dataset Health Team:obs-ux-logs Observability Logs User Experience Team

Comments

@flash1293
Copy link
Contributor

flash1293 commented Apr 11, 2024

In #178954 and #179479 we hid the Estimated data section of the summary because _stats api was not available in serverless - #178955 tracks adding back actual doc size, this will most likely take longer to be resolved.

The ES team is working on enabling a new API to get the count of documents per index in serverless and the work is captured in here.

Once available, this should be used on serverless instead of actual index size to give an indication of data volume.

✔️ Acceptance criteria

  • Estimated data section is shown for serverless deployments with counts of documents instead of size in bytes
  • All e2e tests related to Estimated data section are adjusted accordingly.
  • The tooltip content is changed to The number of documents stored in the last 24 hours.

💡 Implementation hints

@botelastic botelastic bot added the needs-team Issues missing a team label label Apr 11, 2024
@flash1293 flash1293 added Team:obs-ux-logs Observability Logs User Experience Team and removed needs-team Issues missing a team label labels Apr 11, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

@yngrdyn
Copy link
Contributor

yngrdyn commented Apr 22, 2024

Hey @mdbirnstiehl we are changing the Estimated data summary in serverless by Number of documents because size is not available at the moment.

image

What do you think about changing the title of the section to

Number of documents (?)

and the tooltip content to

The number of documents stored in the last 24 hours

?

@mdbirnstiehl
Copy link
Contributor

I think those are fine, with a period at the end of the tool tip.

@gbamparop
Copy link
Contributor

@yngrdyn is this not blocked by https://elasticco.atlassian.net/browse/ES-7700?

@yngrdyn
Copy link
Contributor

yngrdyn commented Apr 24, 2024

@yngrdyn is this not blocked by https://elasticco.atlassian.net/browse/ES-7700?

Partially, I think story is still open because the storage size was removed. The api has been developed including only the doc count https://github.com/elastic/elasticsearch-serverless/pull/1482.

The idea of this story is to replace the Estimated data based on storage size with one based on documents that is the metric available for serverless at the moment.

@mohamedhamed-ahmed
Copy link
Contributor

some quick points here, I would suggest we wait on this until #182873 is merged.

The tooltip content is changed to The number of documents stored in the last 24 hours.

I would suggest we then make this The number of documents stored in the selected time range. as the Estimated Data panel will be reactive at this point.

one more thing that will be affected is the size column, since the size column is now hidden on serverless as well as part of the above linked PR, we will need to modify it so that it also shows the number of documents instead of the size of the dataset.

mohamedhamed-ahmed added a commit that referenced this issue May 14, 2024
closes #182873

## 📝  Summary
After implementing #179172 its
now time to make the `Estimated Data` panel reactive to all table
filters as well.

This PR is the last step into making the summary panel reactive to all
table filters. As part of the implementation the following has been
done:

1. The summary panel state machine is completely removed as its no
longer needed and we now rely on the dataset quality state machine.
2. The Estimated Size panel is now reactive to all filters and show a
sum of the dataset sizes displayed in the table.
3. The `Size` column now shows the size for each dataset within the
given time range.
4. The `Size` column and the `Estimated data` panel are now both hidden
on `serverless` as they both rely on the `stats` API which is not yet
available on serverless.
5. As part of this ticket
#180582 we should start showing
the number of docs instead of the size on serverless.

## 🎥 Demo


https://github.com/elastic/kibana/assets/11225826/a4a4e013-e514-480c-aa4e-eb9d6ff10b82

---------

Co-authored-by: kibanamachine <[email protected]>
@mohamedhamed-ahmed
Copy link
Contributor

@yngrdyn @gbamparop this can be closed now right?

@gbamparop
Copy link
Contributor

@yngrdyn @gbamparop this can be closed now right?

I think so as the estimated size is now displayed in Serverless, @flash1293 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Dataset Health Team:obs-ux-logs Observability Logs User Experience Team
Projects
None yet
Development

No branches or pull requests

6 participants