Live visualization - Websockets / Kafka Subscription

[markrey]

Hi Everyone,

We are successfully using Superset for one POC. Now there is a need of real-time or live visualization via Websockets or Kafka topic.

The data which is coming in gets updated every 100 ms ( Metric is amount of errors in the web application / systems ).

Any suggestions on how I can make live updating dashboard by just Websockets or Kafka subscription?

Thanks,
Mark

[alanmcruickshank]

I think this would have to be a new kind of datasource type alongside druid and sqlalchemy and if you wanted slices to auto update off the feed three interface for slices and dashboards would have to be adjusted to account for that. I'm not sure if this much of a paradigm shift is going to be a useful step for the project. I wonder if tools like statsd and graphite might be more suitable in the mean time?

On Mon, 16 Jan 2017, 17:47 markrey, ***@***.***> wrote: Hi Everyone, We are successfully using Superset for one POC. Now there is a need of real-time or live visualization via Websockets or Kafka topic. The data which is coming in gets updated every 100 ms ( Metric is amount of errors in the web application / systems ). Any suggestions on how I can make live updating dashboard by just Websockets or Kafka subscription? Thanks, Mark — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1990>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdFuMOa94mU75wGYg5IOY1fj6hqUJeAks5rS60UgaJpZM4Lk0xE> .

--

…

_______ Alan Cruickshank E: [email protected] T: +44 770 6851825

[xrmx]

Possibly worth taking this into account in #1995

[markrey]

@alanmcruickshank @xrmx Thank you very much for the response!
I don't have druid cluster running. Would druid be fast enough to refresh every 1 second ? ( without huge latency on loading screen )

@xrmx Actually Apache Drill would be a great addon. I will trace the same.

[QiXuanWang]

Very interesting topic. I saw from other issues that superset can support auto-refresh at a 5 second level.
So is it because of SQLAlchemy/Druid that limits the real-time performance?
I spent some more time on Graphite and Grafana and they looks promising. I really can't say it's so different from superset though. I'll keep watching

[markrey]

One of the way i found is "Kafka -> PrestoDb -> Superset".

But there is nothing "near relatime" in it :)
Also couple of issues on the field identifications as well.

[QiXuanWang]

My application (still trying on superset) currently would accept a 5 second or larger refreshing. But real time is definitely what I would like to welcome and use if that's possible. Let me know or update here if you have any progress. Really appreciated :)

[mistercrunch]

Notice: this issue has been closed because it has been inactive for 430 days. Feel free to comment and request for this issue to be reopened.