Heatmap y-axis sorting logic broken

[nicholaslimck]

Bug description

Observed on Superset 4.1.1, deployed on Kubernetes using the Helm chart.

The Echarts-based Heatmap chart seems to sort the Y-axis through some unknown logic.

A similar issue was raised for the legacy Heatmap chart at #27357 and was fixed in PR #27360. The legacy Heatmap was sorting numeric values as strings, but this behaviour seems to be a bit different. Regardless of whether there is a string or numeric value on the Y-axis, sorting does not seem to make sense as it does not seem to be in alphabetical or numeric order.

X-axis sorting seems to work with my timestamp column both in ascending and descending order.

Screenshots/recordings

With Y-axis sorted by 'Axis ascending':

String values on Y-axis	Numeric values on Y-axis

Superset version

4.1.1

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

@nicholaslimck Hello! I'm here to help you with your Superset issue. I can assist with bug fixes, answer your questions, and guide you on becoming a contributor. Let's work together to resolve the problem you're experiencing.

The Y-axis sorting logic in the Echarts-based Heatmap chart in Superset 4.1.1 is controlled by the sort_y_axis parameter. The available sorting options are alpha_asc (Axis ascending), alpha_desc (Axis descending), value_asc (Metric ascending), and value_desc (Metric descending), with the default being alpha_asc ^[1].

For alpha_asc and alpha_desc, the sorting is done alphabetically using a comparison function. Specifically, alpha_asc sorts the Y-axis values in ascending alphabetical order, while alpha_desc sorts them in descending order by reversing the result of the ascending sort ^[2].

If the sorting does not appear to be working as expected, especially with mixed data types (strings and numbers), it might be due to how the sorting logic handles these types. The current implementation may not correctly handle mixed data types, leading to unexpected sorting behavior. This could be a point of investigation or improvement in the codebase.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[nicholaslimck]

@dosu I don't think the Echarts-based Heatmap uses the same sorting logic as the legacy Heatmap chart. Is the sorting logic inherited from the Echarts project or is it defined within Superset?

[dosubot]

The sorting logic for the Y-axis in the Echarts-based Heatmap chart is defined within Superset, not inherited from the Echarts project. The sort_y_axis control is configured in Superset with specific sorting options (alpha_asc, alpha_desc, value_asc, value_desc) in the controlPanel.tsx file. Additionally, the buildQuery.ts file specifies how the Y-axis is sorted based on the sort_y_axis value, confirming that the sorting logic is implemented in Superset ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[nicholaslimck]

@dosu The sorting logic seems to only consider the query results. Given the following data:

timestamp	machine	SUM(Machine Idle Time)
2024-12-08 20:00:00.000	HTTEX	3403000
2024-12-08 20:00:00.000	M02	5005000
2024-12-08 20:00:00.000	WTTEX	3368000
2024-12-08 21:00:00.000	M02	3895000
2024-12-08 22:00:00.000	CWTEX	613000
2024-12-08 22:00:00.000	HTTEX	3787000
2024-12-08 22:00:00.000	M02	5683000
2024-12-08 23:00:00.000	HTTEX	2055000
2024-12-08 23:00:00.000	M02	3473000
2024-12-09 01:00:00.000	M03	3461000
2024-12-09 01:00:00.000	SAILA06	3431000
2024-12-09 05:00:00.000	HTTEX	2389000
2024-12-09 06:00:00.000	HTTEX	547000
2024-12-09 08:00:00.000	HTTEX	3543000
2024-12-09 09:00:00.000	Assembly Station 1	557000
2024-12-09 09:00:00.000	Assembly Station 2	913000
2024-12-09 09:00:00.000	M02	3464000
2024-12-09 09:00:00.000	M03	1922000
2024-12-09 09:00:00.000	SAILA03	1108000
2024-12-09 09:00:00.000	SAILA04	1090000
2024-12-09 09:00:00.000	SAILA06	718000
2024-12-09 10:00:00.000	Assembly Station 2	1265000
2024-12-09 10:00:00.000	Assembly Station 3	579000
2024-12-09 10:00:00.000	ERVO1	3007000
2024-12-09 10:00:00.000	ERVO2	1565000
2024-12-09 10:00:00.000	ERVO3	1559000
2024-12-09 10:00:00.000	ERVO4	1439000
2024-12-09 10:00:00.000	ERVO5	1221000
2024-12-09 10:00:00.000	M01	2437000
2024-12-09 10:00:00.000	M02	4573000
2024-12-09 10:00:00.000	M03	2238000
2024-12-09 10:00:00.000	SAILA03	543000
2024-12-09 10:00:00.000	SAILA04	1131000

The resultant Heatmap looks like this:

The Y axis is sorted by the value in the 'machine' column, but the sorting is dependent on the timestamp order instead of purely in alphabetical order. This results in a staggered sort effect, where the 'machine' values are sorted on the Y axis based on:

1. Order in which they appear over time
2. Alphabetical order within the same timeframe

However, I want to sort the Y axis only on the alphabetical order of the 'machine' column.

[dosubot]

To sort the Y-axis of the Heatmap chart in Superset based solely on the alphabetical order of the 'machine' column, you should set the sort_y_axis control to alpha_asc. This configuration ensures that the Y-axis is sorted alphabetically, independent of the timestamp order ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[nicholaslimck]

@michael-s-molina I noticed that the legacy Heatmap chart is scheduled for removal in Superset 5.0. Probably a good idea to make sure that feature parity is maintained since the legacy Heatmap's sorting logic was fixed.

I couldn't figure out if there was any other sorting logic applied outside of the query so hopefully someone more familiar with the implementation could take a look.

[michael-s-molina]

Hi @nicholaslimck. I was able to reproduce the problem and I'll submit a fix before 5.0. Thank you for the issue and for adding a dataset example.