[Lens] Improve unclear UI for bucket aggregation grouping order

[mbondyra]

TODO:

• copy review (pending on @AlonaNadler)

Current designs agreed with @cchaos (as of 2 Oct):
in case of two bucket aggregations:

in case of three or more bucket aggregations:

[cchaos]

Here's a diagram of how I've thought through this problem based only on the information provided in the PR summary.

Whimsical link

Essentially this boils down ordering between using the Bucket dimension as the first agg or the Split by dimension as the first agg. Sometimes the data is completely different sometimes it's just an order difference (but the user doesn't need to know these details up front, they just want to see how it gets interpreted on the chart).

The main problem I have with the current UI is that it takes a lot of reading of each option and correlating that option to the different configs. So I think the best approach is to rely on the visual order of the dimensions.

In a typical scenario where there can be only 1 bucket (x-axis) and 1 split, We should add a toggle to the Split by configuration asking the user if they actually want this to be the top level (first) aggregation.

In the scenario where there can be multiple buckets, utilize drag and drop to allow re-ordering of the configs in the dimension so that visually, from top to bottom, is the order the aggregations will be handled.

I know you'll say I'm not accounting for a particular scenario. This is what I've come up with based on all the information provided in this PR summary. For the unnaccounted for scenario, can you find a way to fit it within this concept?

[mbondyra]

Thanks for your work @cchaos! I definitely see this as an improvement and I see your designs as much more understandable.

When it comes to 'unnaccounted' scenario, all I can think of is if we have visualisation that has multiple break downs and x-axis. In this case the multiple break downs grouping would be determined by the order. But then imagine we toggle 'Use as top level aggregation' just for the first breakdown. The order of aggs would be:

• break down 1
• x-axis
• break down 2
• break down 3

and it would be difficult to guess from the UX. On the other hand, we don't have visualizations like these yet. Before I'll implement the above designs, I would like to hear other voices, @flash1293 @dej611 @wylieconlon @AlonaNadler

[flash1293]

I like the toggle as a UI component, but I think the most important part is how this option will be worded.
"Use as top level aggregation" seems problematic to me - we are not talking about aggregations anywhere else in the UI and "top level" is unclear IMHO without knowledge about how Elasticsearch works.

In the scenario where there can be multiple buckets, utilize drag and drop to allow re-ordering of the configs in the dimension so that visually, from top to bottom, is the order the aggregations will be handled.

I'm fine with this, but it would mean this change is blocked by #67226 because we would take away the ability to reorder columns in a table without completely recreating all of them which doesn't seem like a good approach. An alternative would be to show the toggle if there are only two buckets and switch to the current select based UI for >= 3 buckets

I guess we can use that general approach for now, but it won't cover cases with more than two buckets which can't be reordered using drag and drop in the UI like the one @mbondyra mentioned - another one we will run into pretty quickly is having a "split chart" dimension which has the same problem (there will be "x axis", "break down by" and "split chart", and the aggregation order can be any possible combination). So that would mean we definitely have to rethink the solution mid-term (but not within the "Lens by default" timeframe) - up for discussion whether this tradeoff is worth it.

[mbondyra]

Oh yes, @flash1293 I understood it as for now, for more than 2 buckets we're staying with the current UI (select list) and then in the future, with changes in drag and drop we'll remove it.

I'll add this issue for discussion during weekly.

[mbondyra]

Alright, here's what I propose for now:

• 2 or more aggregations in one group -> NOW: Changes are made by a select list. Future: Changes are made by dnd.
• two aggregations in 2 groups -> NOW: Changes are made by a toggle button.
• more than 2 aggregations in more than 1 group (eg 2 aggregations in group x-Axis, 1 in split series) - NOW: Changes are made by a select list. Future: Changes are made by dnd. This leaves us with a problem when we mix the order of elements from two or more groups. Should we stick to the select list in this case?

[flash1293]

I think this is a nice way of handling it, @mbondyra
@cchaos what do you think? About the label for the toggle I would propose Overall <field name> (like done now with the radio buttons, but just showing a single option)

[AlonaNadler]

Thanks for raising and thinking about it @mbondyra
Overall:

• pie/donut/treemap and time series work well
• for table, why do we need the drop down? why not account for the order by the order of columns?

I had this problem with many customers and it's a cause for frequent confusion (in general in any visualization tool). The best way I found so far to explain the sedalty difference is with a descriptive sentence which uses the fields and values. For example:
Top 5 countries by sum(bytes) over time Vs Top 5 countries by sum(bytes) for every hour.

Lens should continue to help with descriptive sentence in the configurator and suggestions description.

In a typical scenario where there can be only 1 bucket (x-axis) and 1 split, We should add a toggle to the Split by configuration asking the user if they actually want this to be the top level (first) aggregation.

having one bucket might be temporary, for example in the future Lens will have more than once breakdown by. Lens will have the ability to do breakdown by another field (another split series or split chart) so I prefer to find an approach that can serve us in the long term.

[flash1293]

@AlonaNadler

for table, why do we need the drop down? why not account for the order by the order of columns?

For now we need the dropdown because we haven't implemented reordering by drag-n-drop

having one bucket might be temporary, for example in the future Lens will have more than once breakdown by. Lens will have the ability to do breakdown by another field (another split series or split chart) so I prefer to find an approach that can serve us in the long term.

+1, we definitely need a UI for the more complex cases (even if we hide it somewhere). Maybe a view with all buckets similar to how Visualize shows it just for reordering (without the edits and "Add" button and with better wording to make clear whats going on)?

I suggest splitting out this case and discussing it separately as we don't have that problem in the Lens by default timeframe and go forward with this PR in the short-term to improve the current situation

[flash1293]

Code LGTM, this leaves the question about wording of the toggle.

Proposal: "Group by < current column name > first".

So, if you have a date histogram on timestamp and a top values on abc, it will say
"Group by timestamp first" if looking at the date histogram and "Group by abc first" if looking at the top values agg.

We already call the section "Grouping", an grouping by an operation "first" is easy to understand IMHO - this will be the first thing that happens, then the other "grouping" will happen. It will probably take a second the first time, but it seems like a consistent concept.

[mbondyra]

Thanks @flash1293 for proposal. I like it! Just to clarify, the copy in this case would be:

1. for top values of category -> "group by category first"
2. for date on timestamp -> "group by timestamp first"
3. for filters -> "group by filters first" (not clear if we have 2 filters aggregations)
4. for intervals on field price-> 1. "group by price" OR 2. "group by intervals" (2. not clear if we have two intervals aggregations)

Because of the confusion in case if we have two filters aggregations, maybe we could somehow indicate that we want to 'group by this field first' or group by this aggregation first ?

[flash1293]

@mbondyra I see this happening but I don't think it's a large issue. If there are two columns named identically, the user will definitely change the labels of at least one. Then it becomes unambiguous again.

I would always use the label of the column, not the field/operation name

[mbondyra]

@elasticmachine merge upstream

[mbondyra]

@elasticmachine merge upstream

[mbondyra]

Got it, here are examples:

[flash1293]

Looks good, I wonder whether we should show the column name in a different font/style to emphasize it's the column name. @cchaos wdyt?

[cchaos]

My main suggestion would be to try to remove some of the receptiveness in order to make the text more scannable. We can add a further description in a tooltip if necessary. But we don't need the word "Group" twice, and I don't think we need to repeat the selected field name especially when field names can be really long and would then increase the size of this label a lot.

My suggestion is just going with this code format for the toggle form layout and some generic wording like "Group by this field first" (italics shown just to emphasize my idea not actual design).

<EuiFormRow display="columnCompressedSwitch" label="Switch">
        <EuiSwitch
          showLabel={false}
          label="Switch"
          name="switch"
          checked={isSwitchChecked}
          onChange={onSwitchChange}
          compressed
        />
</EuiFormRow>

[AlonaNadler]

Question : If we had a way to reorder fields inside a dimension (besides accessibility) do we still need this change in the sorting order?

For table, pie, donut, treemap the ability to order inside the dimension should determine the order of aggregation. Its what users are used to from spreadsheets and we should continue this pattern.

So if I understand it correctly the problem is with xy charts. It doesn't really matter which types of aggregation the users choose whether its filter, histogram or top values. The order ES aggregates between x-axis and breakdown by is what we are trying to optimize here.

[mbondyra]

@AlonaNadler I am not exactly sure I understand what you're asking here – yes, you're right, if we only had one dimension for bucket aggregations this wouldn't be a problem and column order would indicate the grouping – and we would show select list for every chart with plans of replacing it to drag and drop.
And yes, you're right that currently the problem we're trying to solve with the switch only exists in xy charts. But it will surface again with the other groups that we're planning in the future (eg. split charts).

[flash1293]

Tested and LGTM - depending on the case this is not much clearer than the previous copy, but it's much shorter and not worse in any case IMHO.

This is not a final solution, but it evens out some edge cases with histograms and filters in a nice way for now. We will address this again when the case will come up with new chart features.

[mbondyra]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 51792f6

Metrics [docs]

async chunks size

id	before	after	diff
lens	1.1MB	1.1MB	-2.3KB

History

• 💔 Build #80219 failed dcab9e5
• 💚 Build #80020 succeeded 907860e
• 💔 Build #80015 failed 8524824
• 💚 Build #79510 succeeded a4ef511
• 💚 Build #79403 succeeded 153e071

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream