Embedding with buffers

[maartenbreddels]

@jasongrout @SylvainCorlay
Since buffers can be part of the state, we need to think how they will be stored in the json for embedding.
For the js and the py side we can simply use the same code we use to pluck out and insert the buffers, it's only a matter of how to store it in the json state.
I suggest sth like this (looking at the schema), by example:

...
state: {
  description: ...
  buffers:  [
   [['MODEL...', 'bla', 1], {'encoding: ['base64'], data: 'AF... 33'}],
   [['MODEL...., 'lala', 'x'], {'encoding: [], url: 'MODEL_123_lala_x.dat'}]
 ]
}

so it's a list with [path, data object] pairs, where data object specified how it is encoded (as a list, so it can be first zipped, then base64'ed). Or it could be a url, this would allow to load binary files efficiently, since a webserver can zip it on the fly. The only issue would be, can you from a ajax request get a Buffer object?

[SylvainCorlay]

I would say go with base64 encoding of the raw buffer. The memory overhead is only 33%.

The extra complexity of a companion binary datafile does not seem to be worth the effort.

[SylvainCorlay]

There was a very strong case for real binary in the wire format compare to base64 because we could skip the serialization stage and go faster, but in the case of the serialized widget state, it is only a one-off.

[maartenbreddels]

I'm more worried about the performance, a 100mb base64 json will be an performance issue (it will crash the browser), a 100mb raw file that will be loaded similar as now over the websocket will be quite fast.

[maartenbreddels]

We could leave the external files to 7.1, but for ipyvolume it would be useful. But lets do base64 first, but I'd be explicit with the encoding, and put it in.

[jasongrout]

I'm more worried about the performance, a 100mb base64 json will be an performance issue (it will crash the browser), a 100mb raw file that will be loaded similar as now over the websocket will be quite fast.

Right now copying and pasting 100mb of data from a text box would be prohibitive anyway (thinking of how we currently do the 'embed this widget' bit). I think this base64 solution really only works for much smaller data sets (e.g., the data in a simple bqplot chart).

It would be good to iterate on how to do larger data sets. You have some interesting ideas about how to support that at the widget protocol layer. I was thinking we'd do it at the widget serialization layer.