Extend InteractiveImage to work with bokeh server

[DavidVillero]

Awesome library, I really love it.
I usually embed bokeh with flask and it seems to work with datashader as well. However, I noticed that It doesn't do very well when It tries to re-render the image when zooming in. It's because the js code needs the IPython.notebook.kernel for it to work. Is there a way to make it work without the use of the IPython kernel?

[jbednar]

Glad you're finding it useful! Which js code do you mean? If it's the code associated with InteractiveImage, then yes, we do expect to be able to make it work without IPython, but haven't yet done so. Once we do that, we're planning to move InteractiveImage into Bokeh itself, since that code isn't specific to datashader; it will work with any function that can generate an image. If it's some other js code, let me know...

[DavidVillero]

Thank you for the quick response. Yes, That is the one I meant, and it's great to hear that it will be incorporated within bokeh.

[jbednar]

I wish I could give you a timeline on when we'll get to that, but the people here who have the necessary expertise are all heavily overbooked, so it could be a while. If you need this urgently and have the appropriate background, we'd love to see a pull request for a version that works with Bokeh server rather than IPython...

[mpelko]

Hi @jbednar, great work. Just looking into "big" geodata visualizations and Datashader really looks promising. +1 for making InteractiveImage working with Bokeh-server.

[jbednar]

Hi Miha, long time no see. I didn't know you were working in geo-related areas now...

[brendancol]

It's possible to achieve the same ends as InteractiveImage with Bokeh-server using the ImageRGBA directly which is demonstrated in the streaming.py example:
https://github.com/bokeh/datashader/blob/master/examples/streaming.py#L58-L87

Note that updating the client side dimensions:
https://github.com/bokeh/datashader/blob/master/examples/streaming.py#L111-L130

[jbednar]

How long would it take you to make a Bokeh server version of InteractiveImage (or ideally, make InteractiveImage work in either context)? If that's a half-day's work it seems like a good idea. I know it would take me longer than that!

[brendancol]

I think it would just be wrapping the js-dimensions stuff and handling data source updates. I think adding a StreamingImage class bokeh_ext.py would be good, which would wrap the CustomJS from streaming.py and trigger updates on either pan/range/zoom change events (aka InteractiveImage).

[jbednar]

Can one class handle it all, i.e. pan/range/zoom plus trigger for streaming?

[brendancol]

the client-side dimension handling piece will be going away with upcoming bokeh changes, so right now we just need to stick it so folks wouldn't have to deal with the js. I can put something together quick to see what the api would look like

[jbednar]

Is there a chance the bokeh changes will be on master soon, so that we could provide this functionality in a simple way and just say that it currently depends on bokeh master? I don't think it's so urgent we need to fake it, unless it will be weeks before bokeh master will do the job for us...

[bkueng]

Hi
I'm planning to do zoomable time series web plotting for drones flight log analysis, and bokeh looks like great project so far for this. However there are generally too many datapoints, so I need dynamic downsampling on the server. When searching, I came across this issue, and I want to ask what's the status on this?

[jbednar]

The description above is still correct; we are still planning to extend it to work with the separate server, and we still haven't done it! :-/ We're actually planning to try it out in the next couple of weeks, but we may hit some snag, so I can't promise anything.

[jbednar]

Meanwhile, it should work fine in the notebook, and you can try translating the approach to work with Bokeh Server; what we're hoping to do instead is to have a single class work smoothly in both cases. But if you don't need that, you can already use the notebook, and making a separate version for Bokeh server shouldn't be that hard.

[bkueng]

ok, thanks for the prompt reply. I'll look into it

[bkueng]

I implemented a downsampling class, based on the x_range callback for bokeh server:

class DynamicDownsample:
    def __init__(self, bokeh_plot, data_source, x_key):
        """ Initialize and setup callback

        Args:
            bokeh_plot (bokeh.plotting.figure) : plot for downsampling
            data_source (bokeh.models.ColumnDataSource) : data source of the
                plots, contains all samples. Arrays are expected to be numpy
            x_key (str): key for x axis in data_source
        """
        self.bokeh_plot = bokeh_plot
        self.x_key = x_key
        self.data_source = data_source

        # parameters
        # minimum number of samples/pixel. Below that, we load new data
        self.min_density = 2 #3
        # when loading new data, number of samples/pixel is set to this value
        self.init_density = 5 #6
        # when loading new data, add a percentage of data on both sides
        self.range_margin = 0.2

        # create a copy of the initial data
        self.init_data = {}
        self.cur_data = {}
        for k in data_source.data:
            self.init_data[k] = data_source.data[k]
            self.cur_data[k] = data_source.data[k]

        # first downsampling
        self.downsample(self.cur_data, self.bokeh_plot.plot_width *
                self.init_density)
        self.data_source.data = self.cur_data

        # register the callbacks
        bokeh_plot.x_range.on_change('start', self.x_range_change_cb)
        bokeh_plot.x_range.on_change('end', self.x_range_change_cb)


    def x_range_change_cb(self, attr, old, new):
        cb_start_time = timer()

        new_range = [self.bokeh_plot.x_range.start, self.bokeh_plot.x_range.end]
        if None in new_range:
            return
        plot_width = self.bokeh_plot.plot_width
        init_x = self.init_data[self.x_key]
        cur_x = self.cur_data[self.x_key]
        cur_range = [cur_x[0], cur_x[-1]]

        need_update = False
        if (new_range[0] < cur_range[0] and cur_range[0] > init_x[0]) or \
                (new_range[1] > cur_range[1] and cur_range[1] < init_x[-1]):
            need_update = True # zooming out / panning

        visible_points = ((new_range[0] < cur_x) & (cur_x < new_range[1])).sum()
        if visible_points / plot_width < self.min_density:
            visible_points_all_data = ((new_range[0] < init_x) & (init_x < new_range[1])).sum()
            if (visible_points_all_data > visible_points):
                need_update = True
            # else: reached maximum zoom level

        if visible_points / plot_width > self.init_density * 3:
            # mostly a precaution, the panning case above catches most cases
            need_update = True

        if need_update:
            drange = new_range[1] - new_range[0]
            new_range[0] -= drange * self.range_margin
            new_range[1] += drange * self.range_margin
            num_data_points = plot_width * self.init_density * (1 + 2*self.range_margin)
            indices = np.logical_and(init_x > new_range[0], init_x < new_range[1])

            self.cur_data = {}
            for k in self.init_data:
                self.cur_data[k] = self.init_data[k][indices]

            # downsample
            self.downsample(self.cur_data, num_data_points)

            self.data_source.data = self.cur_data

It uses the most straight-forward downsampling, which certainly could be improved. It already works really well, even with a simulated low bandwidth connection. In this case, requests tend to queue up, which I'd like to avoid, by adding timeouts, as InteractiveImage does. Also I'd like to move some checks to the client-side to avoid unnecessary server requests. How can I do that, I did not see how to do a callback to the server from CustomJS?

[jbednar]

I'm not sure if you've seen the latest version of InteractiveImage (in the past few days) which had significant improvements to how the events are queued. If you haven't, you should check it out. @philippjfr, can you look at the code above and think about whether something like that would work for a Bokeh equivalent to InteractiveImage?

[bkueng]

Yes I've seen that, that's pretty much what I want. However it uses the notebook IPython.notebook.kernel.execute to callback into python, which cannot be used for the server.

[tangfucius]

Is there any updates to this? Would be nice to be able to use InteractiveImage in Bokeh Server instead of writing my own version...

[jbednar]

Sorry; we haven't had a chance to look into this yet. Instead, we've been focusing on adding datashader support to HoloViews (see holoviews.org), which allows plots to be constructed very easily now, mixing different options for aggregating, decimating, etc. as needed while still using Bokeh as the backend. I think the plan would be to release that support first, which offers a much higher-level interface for working with this type of plotting, and then afterwards think about how specific bits of that can be made available for pure bokeh plots without HoloViews.

[stevievb]

Has there been any progress on this? I am trying to run bokeh on a backend webserver and have it connected to plots in a web page with the downsampling functionality. Would this be the right approach?

@bkueng could you post your self.downsample() code? Are you just using your DynamicDownsample class in place of InteractiveImage?

[jbednar]

No one is currently working on improving InteractiveImage, but the datashader support in HoloViews is now very mature, which makes it trivial to mix and match Datashader and regular Bokeh-based plots in any combination. We're now waiting on having full Bokeh Server support in HoloViews, which is partially working but still needs some attention.

[stevievb]

Ok thanks, so this approach is somewhat of a dead end it sounds like.

I took a look at #694 and even tried messing with the bokeh_server branch of holoviews but it seemed like the holoviews code is pretty wrapped up in the IPython kernel. Is the idea that at some point you could use holoviews, bokeh, and the bokeh server with no IPython or is that always going to be required?

[philippjfr]

Is the idea that at some point you could use holoviews, bokeh, and the bokeh server with no IPython or is that always going to be required?

Yes that's the idea, there's a WIP PR here that allows HoloViews to work directly with server. A minimal datashading bokeh app will look something like this:

import numpy as np
import holoviews as hv
import holoviews.plotting.bokeh
from holoviews.operation.datashader import datashade

hv.Store.current_backend = 'bokeh'
renderer = hv.Store.renderers['bokeh'].instance(mode='server', holomap='server')
points = hv.HoloMap({i: hv.Points(np.random.multivariate_normal((0,0), [[0.1, 0.1], [0.1, 1.0]], (1000000,))) for i in range(10)})
datashaded = datashade(points, x_sampling=0.01, y_sampling=0.01)
doc, _ = renderer(datashaded)
doc.title = 'HoloViews Datashade'

We're hoping to release HoloViews 1.7 in mid-February, after which I will focus on getting the server support integrated.

[jbednar]

so this approach is somewhat of a dead end it sounds like.

Technologically it's not a dead end; I think it's very reasonable to have something like InteractiveImage for Bokeh Server, and I would be happy to have that in the library. But that depends on someone actually building it, and it's not on the roadmap for any of our developers right now. The reason it's not on the roadmap is that even if it does exist, it will probably have all the limitations that InteractiveImage has in the notebook -- you won't be able to have multiple such plots in the same application, etc. And once you've addressed all the issues like that, you pretty much end up with HoloViews, hence our focus on making the HoloVIews support work first, to solve all problems like this.

Once the HoloViews approach works well, it would make good sense to provide something intermediate for Bokeh Server that works well enough for many cases without making people learn how to use HoloViews if they are already comfortable with Bokeh itself. So if anyone wants to work on that now, great; otherwise we may get to it, but definitely not for the next few months.

[stevievb]

Thanks so much for the quick and helpful responses.

I think this technology is really great and I want to include it in my project (also I don't know of any other way to plot 10 million points in a browser), but I of course need to get something working sooner then later, even if it's very rough.

It seems like I have two options to do that. One is to start with the bokeh_server branch of holoviews and the other would be creating an InteractiveImage for bokeh server. As I don't know any of these codebases at this point, in your opinion what do you think would be the better approach to get something working as a proof of principle?

[jbednar]

I'm not sure about your requirements, but personally I'd just make a Jupyter notebook using HoloViews (or InteractiveImage, if it covers your needs), then deploy it using Jupyter Dashboards. I.e., if what you need is to deploy something as a web app, that's already fully supported, just not with Bokeh Server (yet).

[bkueng]

@stevievb My implementation is here: https://github.com/PX4/flight_review/blob/master/plot_app/downsampling.py
I use it with the bokeh server and it works well. The downsampling class is used here: https://github.com/PX4/flight_review/blob/master/plot_app/plotting.py#L392

[stevievb]

@bkueng Thanks for the links. I'm going to give it a try.

@jbednar thanks for the idea. I didn't know about Jupyter dashboards, but unfortunately I don't think that will work for me. I'm running an aiohttp webserver that is sharing its asyncio event loop with the bokeh server. I need to be able to dynamically request plots of different data sets from a database and embed them in an existing front end angular2 web application. I have that all working now, I just need the downsampling functionality to handle some of time series data I have which can contain upwards of 10 million points.

[bryevdv]

I am not fully versed on the Datashader side of things, but I should mention that in 0.12.4, Bokeh added support for getting at a plot's "inner" dimensions, i.e. the area inside the axes where glyphs are actually draw. You can see an example here: https://github.com/bokeh/bokeh/blob/master/examples/plotting/server/responsive.py The range start and end (in data coordinates) have always been available. As far as I understand, these are all the dimensions that are needed to feed into a Datashader pipeline to generate an image of the appropriate size. I could be wrong but it seems like an "update" callback the fires whenever any of these change, that creates a new Datashader image, and updates column data source for an Image glyph, might work.

[jbednar]

Right -- those should be very helpful in a Bokeh server equivalent to InteractiveImage. InteractiveImage itself doesn't use them yet, as it will need special code to fetch those dimensions over from the JavaScript side. For Bokeh Server I believe that's handled for you (as shown in the inner-dimensions example linked to the most recent Bokeh release notes).

[jnettels]

Hi there,
I am curious if there are any updates on this issue.
Thanks for your great work!

[jbednar]

We don't currently have any plans to expand InteractiveImage, because the Datashader support provided by HoloViews and documented at the Datashader and HoloViews sites covers all of our expected use cases for InteractiveImage on both Bokeh Server and in Jupyter notebooks. Thus we are unlikely to work on this issue, and I'll close it, but if someone wants to use the fact that Bokeh now has more similar communication mechanisms in the notebook and server contexts, it might not be a big project anymore.