[SIP-50] Proposal for using ECharts as our main charting library

[mistercrunch]

Motivation

When Superset was started back in 2015, the d3 library was the clear choice for building data visualizations, and nvd3 offered a nice set of reusable charts built on top of d3 that allowed hooks and callbacks that integrated nicely with d3 primitives. Over the past few years, the library has not been maintained actively and we've had to fork the project recently to ensure we could fix some bugs and to put it on life support. We got a lot of mileage out of the library and we're grateful for that, but it's beyond time to move on.

Superset has also grown to support 30+ visualizations that consists of a patchwork of libraries that are rooted in a vast dependency tree. This creates visual inconsistencies and many challenges around keeping up with different projects, people, APIs, code styles, levels of quality, ...

Looking at the vast array of choice in the visualization libraries space here are some selection criteria:

• a vast library that offers a wide array of visualization types, but are consistent from a visual and an API standpoint
• a thriving, active and growing community
• a good governance model where we can get involved and have a path to contribute and influence direction
• a rich set of visualization primitives that enable extensibility
• performance: support for canvas rendering or alike
• "themeable"
• i18n support

Proposed Change

Standardize on using ECharts! as our primary visualization library for core visualization that ship with Superset.

This library checks all the boxes stated above and more:

• one of the most vast library out there
• governance: ECharts is also incubating at the Apache Software Foundation
• we established contact with the team and they are open to collaborate - both sides sees a lot of synergy potential

New or Changed Public Interfaces

We'll be leveraging the plugin interface that has recently settled.

New dependencies

The ECharts library is surprisingly nimble and only depends on its rendering engine zrender which is managed and maintained by the same group of people. In the longer run, this could really help simplify our dependency tree and the risks that come with that, and perhaps help reduce our bundle sizes.

Migration Plan and Compatibility

The idea is to create new visualizations plugin that use ECharts and to progressively replace the existing visualizations with ECharts-powered visualizations. We may use one or many feature flag or other mechanisms to let administrator decide which set of charts they want to use in their environments while ECharts visualizations are maturing. We should also offer a migration path (through a database migration) to help our users migrate to the new core charts as they become core.

Rejected Alternatives

The hundreds of other amazing chart libraries out there. These can still be developed as plugins or plugins pack and shared as part of our growing plugin ecosystem. While we welcome for different plugins and plugin packs to be developed and shared by the community, this SIP is about setting the vision for the future of the core libraries that ship with Superset.

[issue-label-bot]

Issue-Label Bot is automatically applying the label #enhancement to this issue, with a confidence of 0.86. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[dingsheng999888]

Very good!This is what I want.

[zcfrank1st]

I just developped myself using echarts

[carstenkolve]

This looks like a very good move - echarts looks very impressive and offers a wealth of viz options!
Two questions (please excuse if the answers are obvious, I am very new to using superset)
a) echarts does not seem to offer any dedicated visualizations for tables / pivot- tables ; is there a plan to leverage other libraries to improve this kind of data display (especially in terms of conditional formatting etc.)
b) will the move enable non temporal x-axis on line/bar charts, or is this an underlying issue not related to the chartiung library itself? #4185

[ktmud]

a) echarts does not seem to offer any dedicated visualizations for tables / pivot- tables ; is there a plan to leverage other libraries to improve this kind of data display (especially in terms of conditional formatting etc.)

We just had a big refactor on table chart and plan to do the same for pivot tables. You may not notice the difference but it paved ways for advanced features such as conditional formatting. No library provides conditional formatting out of the box, the majority of the work lies within Superset itself, so it doesn't really matter which library we use for this specific feature.

b) will the move enable non temporal x-axis on line/bar charts, or is this an underlying issue not related to the charting library itself? #4185

We didn't prioritize non temporal x-axis because we didn't want to keep building new stuff based on nvd3, which we know will soon be deprecated. Once we start migrating line chart to Echarts (or any other better maintained libraries we eventually decide on), we'll be sure to add this to the TODO-list.

[villebro]

b) will the move enable non temporal x-axis on line/bar charts, or is this an underlying issue not related to the chartiung library itself? #4185

No formal decisions have yet been made on what features the new charts will have. But I think we'll be moving in the direction of introducing an ECharts based time series chart, that incorporates the main features of the current line, bar and area chart, also adding some new functionality that doesn't exist in the current charts, like forecasting capabilities. To complement that I think it makes sense to introduce a similar one with a non-temporal x-axis that makes it possible to use categorical or numeric values. However, I stress that we are still in early stages, so this plan might change.

[carstenkolve]

Thanks for the heads up - I appreciate the transparency and quick feedback!

[amitmiran137]

We compared Recharts to Echarts and basically a major difference was that eCharts does not support out of the box for react which means it might not be customisable in all uses-cases bc it is dependent on what the configuration json they support covers

there is a react-wrapper for echarts solution but this is a problematic pattern to begin with

[ktmud]

@amitNielsen while the React API is nice, I don't think Recharts is particularly more customizable than ECharts. What you can do with Recharts is also limited by what API is provided by the React components.

[amitmiran137]

@ktmud not sure what you mean a react component is 100% customisable by definition, no limitation there

[rusackas]

FWIW, I haven't found a need to use the react wrapper for eCharts thus far. A useEffect hook with a ref seems to work just great for Superset's purposes.

It's worth re-mentioning, this SIP is to use ECharts as the main charting library, particularly for the core plugins, but not necessarily the only one on the table for use in Superset. As mentioned in the SIP, this decisoun should not preclude people from creating plugins using alternate libraries. In fact, I'm hyped up about it! We just need to figure out how/where to support those plugins, which is another discussion taking shape on Slack.

[ktmud]

@ktmud not sure what you mean a react component is 100% customisable by definition, no limitation there

I mean you are still limited by what low-level React components Recharts provide, what customization props are available, and how they are assembled by the library while rendering.

[mistercrunch]

VOTING HAS STARTED - see the dev@ mailing list!

[isunix]

Hope we can use ECharts in Superset very soon.

[johnoscott]

If this move means accelerating chart progress then I am for it.

[x670]

Will this library solve epic problem with non temporal line charts?
If it is, I am for it!

[isunix]

When we can use echarts?

[villebro]

@isunix we have already implemented a new ECharts based Timeseries Chart that can do line, area, bar and scatterplot. This is currently only available on master branch, but will be available on the forthcoming 0.38 release. In the next few days we also expect to replace the current pie chart with an ECharts based Pie Chart replacement. Stay tuned for more charts to follow! Also, if you want to help speed up the migration process by collaborating on new charts, don't hesitate to be in touch!

[isunix]

@villebro Thank you for replying to me. Yes we need ECharts very much. May I ask when will the 0.38 be available to us?

[junlincc]

We ran a performance comparison with same dataset in NVD3 and Echarts line chart on the dashboard.
Applying time grain filter to compare
going from time-grain = year (9 rows) -> quarter (33 rows) -> week (2.5k rows) -> no time grain (10k rows)
At 2.5k rows, NVD3 takes much longer to render
At 10k rows, stuck entirely...

By proofing Echarts migration's benefits from ALL angles(functionality, performance, UX and extensibility), we hope to attract more people from the community to contribute and expedite this project.

Thanks for all the hard work of the superset-echarts team for the improvement @villebro @mayurnewase 🙏

cc @mistercrunch @srinify @eugeniamz @zuzana-vej @ktmud

[eugeniamz]

we just need to add the advanced analytic section to the echarts and I like it more than the old one!

[rusackas]

Vote has passed, work is in progress, and therefore I'm marking this issue as closed. Thanks all! Super excited about this!