Docker image : enable extensions

[dbeniamine]

Is your feature request related to a problem? Please describe.

I am trying to use the dockerized version of nbviewer to present some private notebooks (localfiles). Some of them use the widget extension (bokeh and/or interact) but the extension does not seem to work on the docker image : none of my bokeh or interact plots are shown while "normal" matplolib plots appears normally.

Describe the solution you'd like

Install and or setup the widget extension on the nbviewer docker image

Describe alternatives you've considered

I tryed running jupyter nbextension enable --py widgetsnbextension directly on the docker container but it does not work as I am running bash as nobody. To run this command while building the image, I also had to install jupyter via conda but it did not fix my problem ....

[krinsman]

Do the plots render correctly when viewed in Jupyter Notebook?

Or is it only in NBViewer where they are rendered incorrectly?

This seems like it might be a Jupyter notebook or conda environment issue.

Could you attach your environment.yaml file?

[dbeniamine]

Yes they render correctly in the Notebook (after installing the widgetsnbextensions).

I do not see any environement.yaml file inside the docker container ...

[krinsman]

If there were an environment.yaml, it should be the same thing as the output of conda list.

You said "the Docker image" -- which Docker image? Could you include a link to it?

Sorry about the confusion.

And when you say "in the Notebook", do you mean running the Notebook from within the docker container? Or running the notebook elsewhere?

[dbeniamine]

I am using the nbviewer docker image which is built from this repository.

I think the output of conda list will not be useful as the docker image forces me to be nobody when I run docker exec:

nobody@41ee053763fe:/srv/nbviewer$ conda list

EnvironmentLocationNotFound: Not a conda environment: /opt/conda

[krinsman]

OK, I found the Dockerfile here: https://hub.docker.com/r/jupyter/nbviewer/dockerfile

I can't find the corresponding GitHub repo where you could make a Pull Request however.

There isn't a widgetsnbextensions package in the main Anaconda channel seemingly, but there does seem to be a conda-forge one: https://anaconda.org/conda-forge/widgetsnbextension

Looking at the current Dockerfile it seems like it's already downloading packages from conda-forge instead of defaults, which is good. Maybe this might work? Change:

# Python requirements
ADD ./requirements.txt /srv/nbviewer/
RUN conda install --file requirements.txt

to

# Python requirements
ADD ./requirements.txt /srv/nbviewer/
RUN conda install --file requirements.txt && \
conda install widgetsnbextension

Worst case scenario you could also remove the line

USER nobody

and run the container as root (as long as you're just running this on your personal computer and not deploying it publicly).

I am not sure which of the configuration settings are making it impossible to run conda as a regular user -- I thought conda was supposed to work better when not running as root, or that running as root is discouraged?

The webpage says the Dockerfile was updated already 12 days ago, so apparently someone has write access for it and is updating it, but I don't know who, or where (if anywhere) one could make a PR for such things. It doesn't seem like it's in this repository though. I know that's not helpful, I'm sorry.

[krinsman]

Oh I'm so sorry+dumb; here's the Dockerfile, it is in this repository:

https://github.com/jupyter/nbviewer/blob/aa567da928dd022ce6a75b6c131a5ef3ff5c211a/Dockerfile

Anyway try the change

# Python requirements
ADD ./requirements.txt /srv/nbviewer/
RUN conda install --file requirements.txt && \
conda install widgetsnbextension

and if it works, it might make sense to make a PR against this repo with the change to the Dockerfile.

Bokeh is a fairly commonly used library, and it does appear as one of the suggested notebooks on the frontpage, so my personal opinion (I don't have write access to the repo FYI) is that it could make sense to better integrate the Docker image to produce notebooks created with Bokeh.

[dbeniamine]

I tried the above solution but it did not work, there is the output of some tests that may help :

Listing extensions

Command : docker-compose exec nbviewer jupyter nbextension list
Using the base image:

Error executing Jupyter command 'nbextension': [Errno 2] No such file or directory

Same if I remove the USER statement

If I add the conda install widgetsnbextension, I have the following answer :

Known nbextensions:
  config dir: /opt/conda/etc/jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled 
      - Validating: OK

But I still do not see my interactive plots

[krinsman]

Do you not see the plots at all, or are they not interactive? Any Python code in the notebook won't be run, so if the plots require running Python code in order to be interactive, then it's impossible for them to be interactive with NBViewer, since it just converts outputs to HTML (and javascript embedded in HTML too I think).

Are any of your plots in a notebook which is on GitHub? What happens when you copy-paste the URL of that GitHub notebook into nbviewer.jupyter.org? Do the plots work?

It seems like some Bokeh plots should work:
https://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blob/master/tutorial/09%20-%20Geographic%20Plots.ipynb
https://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blob/master/tutorial/10%20-%20Exporting%20and%20Embedding.ipynb

To clarify, when you say they render correctly in Jupyter Notebook, are you running Jupyter Notebook from within the container created by the NBViewer image?

It should be possible to do docker exec -it <name of container> bash in order to enter a terminal inside of the container and run arbitrary commands, including installing Jupyter Notebook.

[dbeniamine]

Ok so I tried the following:

1. Try to rebuild the ipynb out of the container
   1. Execute the notebook locally (on my computer not docker) and export the ipynb, plots are visible and interactive
   2. Copy the ipynb file on a volume of the docker
   3. Try to access it on the docker
   4. Interactive plots are not visible
2. Try to execute jupyter notebook from the container
   • No plots, and when I try to run the cells I've got these errors :

     # Cell content
     %matplotlib inline
     # Error
     ModuleNotFoundError: No module named 'matplotlib'
     # Cell content
     from ipywidgets import interact, interactive
     import matplotlib.pyplot as plt
     import numpy as np # Error
     #Error
     ModuleNotFoundError: No module named 'ipywidgets'

3. Try to access data from github
   1. Using nbviewer.jupyter.org, we can see bokeh interactive plots for instance here
   2. Trying to view the same notebook from my container : I see the bokeh plot
   3. Downloading the ipynb from the above example and putting it on a volume on the container:

      400 : Bad Request
      We couldn't render your notebook
      Perhaps it is not valid JSON, or not the right URL.
      If this should be a working notebook, please let us know.
      The error was: Error reading JSON notebook
      

   4. Put my example on github here
      1. With nbviewer.jupyter.org : still no interactive plots
      2. With my docker container: still no interactive plots
      3. Try to visualize it directly from github : Sorry something went wrong

So at this point it seems clear that there is a problem on my ipynb file ....

Therefore I tried a second, simpler notebook with only a bokeh (interactive with JS) plot and this time I see it in the docker nbviewer !

A last test to be sure that I understand the problem correctly : I add interactive plots using jupyter interact widget in my bokeh.ipynb test, and all plots using this interaction are invisible.

Conclusion : as @krinsman says, interact widget probably requires to run some python code, thus it cannot work on nbviewer, still bokeh is only interacting with JS code, so Bokeh+JS interaction works which is enough for me.

Thanks all for your help !

I will not remove my test repository so it can helps other people experiencing similar issues.

I guess you can close the issue.

[krinsman]

@dbeniamine Regarding the 400: Bad Request error you got: I got something similar once, and what turned out to be the issue I think was that the notebook was old enough that its JSON format wasn't up to date.

See: https://nbformat.readthedocs.io/en/latest/

There are some places (I think JupyterLab but not Jupyter notebook? I don't remember to be honest) where opening an old notebook won't throw an error because it automatically converts the file into the newest JSON format. This would explain at least why creating a new, simpler notebook would work, since any new notebooks would be created with the most recent JSON format.

Regarding why it doesn't work on GitHub -- GitHub scrubs out most/all Javascript in notebooks, and so isn't able to display things fully the way NBViewer can. So NBViewer being able to render notebooks like yours is very important, since it's one of the most noticeable ways in which NBViewer differentiates itself from GitHub's preview.

These issues in ipywidgets might be related/helpful:
jupyter-widgets/ipywidgets#2063
jupyter-widgets/ipywidgets#1458
jupyter-widgets/ipywidgets#318
https://minrk-ipywidgets.readthedocs.io/en/latest/embedding.html#rendering-interactive-widgets-on-nbviewer

But anyway NBViewer is supposed to work with widgets, at least with some help:

nbviewer/nbviewer/app.py

Line 314 in f1c8cb1

define("jupyter_js_widgets_version", default="*", help="Version specifier for jupyter-js-widgets JS package", type=str)

It is good to hear at least that Bokeh seems to be working for you. I apologize that I'm not being more helpful -- I know and understand very little about Bokeh or IPyWidgets, to be perfectly honest.

[dbeniamine]

@krinsman : you found the issue !

If I save the widget state on jupyter notebook, then nbviewer shows them.

Thanks a lot 👍