Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a proper widget to use with notebooks #6518

Closed
jleibs opened this issue Jun 7, 2024 · 1 comment · Fixed by #6573
Closed

Create a proper widget to use with notebooks #6518

jleibs opened this issue Jun 7, 2024 · 1 comment · Fixed by #6573
Assignees
Labels
notebook Jupyter notebooks etc

Comments

@jleibs
Copy link
Member

jleibs commented Jun 7, 2024

Out jupyter notebooks currently use some awkward hacks that involve returning an html snippet containing a base64-encoded RRD, an iframe pointing at a rerun instance, and some javascript to stitch everything together.

Although this is functional, it's sub-optimal for numerous reasons:

  • With hosted notebooks environments, we can run into CORS issues.
  • Attaching the RRD to the DOM chokes when files get to big.
  • The whole snippet has to be created up front and then sent to the viewer rather that incrementally streaming.

All of the original work predates having our rerun-io/web-viewer npm package, and any of the things we've learned about streaming via the gradio interface.

The right thing to do going forward is to create a proper jupyter widget, which would allow the notebook to host the assets (avoiding CORS issues), and tunneling any data through the proper notebook data-transmission mechanism.

I suspect anywidget might be a good candidate for achieving this. It looks like there are even ways to use anywidget with svelte components: https://gist.github.com/manzt/af46972d7a0a8e870f5228da66c52891

@jleibs jleibs added the notebook Jupyter notebooks etc label Jun 7, 2024
@jleibs
Copy link
Member Author

jleibs commented Jun 10, 2024

Proposal for possible steps:

  1. Proof-of-concept anywidget implementation: create a standalone package that bundles rerun-io/web-viewer and can be used as a widget that just loads an empty viewer in a notebook.
  2. Add APIs to that widget that map to the JS APIs for controlling channels and sending partial RRD blobs.
  3. Figure out packaging story -- does this need to be a separate pip package (rerun-notebook)? Are the deps light enough that we can bundle with rerun-sdk or does maturin bundler fight with anywidget bundler?
  4. If we bundle with rererun-sdk update rr.notebook_show() to use the new APIs. Otherwise maybe that equivalent code moves to rerun_notebook.show()? Either way, show will still submit all the data as a single blob, but it will do so by having sent the data over a channel as opposed to the current DOM hacks.
  5. Design new API that separates out the phases of creating the widget (with optional blueprint) from streaming the data. Ideally this could work within a single cell where the Rerun widget appears immediately when cell execution starts and then the data is sent to the widget over the course of cell execution.

@jprochazk jprochazk mentioned this issue Jun 14, 2024
5 tasks
jprochazk added a commit that referenced this issue Jun 24, 2024
### What

- Closes #6518


### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/6573?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/6573?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/6573)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.

---------

Co-authored-by: Jeremy Leibs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
notebook Jupyter notebooks etc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants