💁 API to make objects available inside JS

[fonsp]

Currently, to get data from user Julia to user JavaScript is to use JSON:

let
	x = rand(UInt8, 10_000)
	
	d = Dict(
		"some_raw_data" => x,
		"wow" => 1000,
	)

	HTML("""
	<script>

	let d = $(JSON.json(d))
	console.log(d)

	</script>
	""")
end

However, this has performance problems with large amounts of data:

• Data size increases greatly when stored as decimal strings
• There is unnecessary Float64 -> decimal string -> Float64 conversion
• More data means more javascript code to parse
• etc etc

JSON is especially problematic for raw array types like Vector{UInt8} and Vector{Float64}, which get converted into JS number arrays.

Solution!

We allow users to 'published' objects, which will re-use Pluto's existing data synchronisation mechanism. This system is MsgPack-based, and particularly optimized for Vector{UInt8} and such, and will send those straight from memory. On the JavaScript side, they are exposed as native typed array buffers: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays#typed_array_views and it should have the highest performance possible.

A 'magic' function that makes a given object available inside user JavaScript. The function publish_to_js returns a JavaScript command that can be interpolated into scripts.

let
	x = rand(UInt8, 10_000)
	
	d = Dict(
		"some_raw_data" => x,
		"wow" => 1000,
	)
	
	HTML("""
	<script>
		
	const d = $(PlutoRunner.publish_to_js(d))
	console.log(d)
	
	</script>
	""")
end

The precise JS command returned is an implementation detail, but currently it works like this:

• PlutoRunner.publish_to_js calls PlutoRunner.publish which registers the object on an internal Dict, indexed by its object id. publish returns the object id as 'key'.
• The dict of published objects is part of the cell struct, which gets synchronised with the frontend.
• The JS command accesses the object through an internal function call.
• When the cell changes output or is deleted, old published objects are cleared. If the same object is re-published, then no data needs to be transferred, because Pluto's state synchronisation is based on diffing.

TODO:

• document
• not available on first load
• tests
• think about data urls

[fonsp]

@jheinen @j-fu

[j-fu]

Time for PlutoMakie?
As far as I understand this would now allow to setup a three.js scene with data from Julia?

[lungben]

For future reference, this is how to use it with HypertextLiteral.jl:

using HypertextLiteral: @htl, JavaScript

_transfer_data(data) = if isdefined(Main, :PlutoRunner) && isdefined(Main.PlutoRunner, :publish_to_js)
	# faster data transfer using MsgPack if available
	JavaScript(Main.PlutoRunner.publish_to_js(data))
else
	data
end

@htl("""
<script>
const jsData = $(_transfer_data(data));
</script>
""")

Thanks @disberd !