Jupyter Tutorials

[griegler]

I changed most existing Basic and Advanced tutorials to Jupyter notebooks and integrated them via nbsphinx. This builds up on #1640.

Currently, this has three issues:

1. The references/citations are creating warnings when building the doc. I tried different formats in the notebooks, but they all did not work reliably so far.
2. The stdout of the open3d Log* functions is not visualized in the cell outputs, but is only printed in the terminal running the jupyter instance. I did improve the Logger such that it can redirect the output to Python's stdout. In addition, I added a context manager to simply change the verbosity level for a code block.
3. For some visualizations the default view point is very bad. I will have to extend the draw_geometries function to directly pass a view point for that. I added a customization to the draw_geometries function in the jupyter notebooks. However, those settings are not yet available in the native draw_geometries because I do not know yet how to handle default None parameters in pybind11. @yxlao any ideas.

I have moved/written tutorials for most existing examples in examples/Python/Basic and examples/Python/Advanced. There exist still a few .py files in those folders without proper jupyter notebook tutorials. This can have a few reasons:

1. The code is outdated, e.g., half_edge_mesh.py
2. The code is referenced in some .rst files, e.g., Advanced/*visualization*.py
3. I am not sure where to put a tutorial for some files, e.g., Advanced/trajectory_io.py

Maybe @yxlao @qianyizh @germanros1987 can have a look at the remaining Python examples and add tutorials as needed.

---

This change is 

[update-docs]

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

[benjaminum]

Some notebooks do not work because of missing files (fountain). Do I have to download them somewhere?

[griegler]

@benjaminum Yes, it is stated in the color_map_optimization.ipynb notebook that you have to download the dataset. It is hosted on a Google drive, and I would need to add another Python package to automatically download it, but it would be possible to automate it.

[benjaminum]

OK, then it is just me too lazy to read the instructions.

[benjaminum]

Was able to build all the sphinx docs. Visually they look OK. Having the console output in the notebook is good but sometimes the output is way too long. Generating a scrollbox in the html output would help but I did not find a simple solution online. It feels like we should cope with that but maybe not in this PR.

Reviewed 65 of 85 files at r4, 39 of 39 files at r5.
Reviewable status: 103 of 108 files reviewed, 1 unresolved discussion (waiting on @benjaminum, @griegler, and @yxlao)

---

docs/conf.py, line 212 at r5 (raw file):

autodoc_member_order = "groupwise"

# Copy jupyter notebooks and test data to tutorial folder

Just a question: do you think it is possible to do without copy such that notebooks don't need to be rebuild?
The build times are quite long.

[griegler]

I actually implemented the context manager to only have debug output for selected statements. But yes, sometimes it could still get a bit long. I think we could capture the output and then strip it in another PR.
Regarding build times. If you build the jupyter notebooks first in the examples, then they are not executed in the sphinx script (after copying). Do you have another idea, something else in your mind?

Reviewable status: 103 of 108 files reviewed, 1 unresolved discussion (waiting on @griegler and @yxlao)

[benjaminum]

I think now I understand the workflow for building the notebooks.

Reviewable status: 103 of 108 files reviewed, 1 unresolved discussion (waiting on @griegler and @yxlao)

[yxlao]

1. "It is hosted on a Google drive, and I would need to add another Python package to automatically download it, but it would be possible to automate it."
   • To make it more robust, let's host the datasets in our google cloud bucket under
     • https://storage.googleapis.com/isl-datasets/open3d-dev/path_to_file
   • The fountain dataset has been uploaded to
     • https://storage.googleapis.com/isl-datasets/open3d-dev/fountain.zip
   • I've sent the bucket instructions via Discord to @griegler
2. "However, those settings are not yet available in the native draw_geometries because I do not know yet how to handle default None parameters in pybind11. @yxlao any ideas."
   • Your current approach looks pretty reasonable. To take it a step further, we can bind the C++ code to o3d.visualization._internal_draw_geometries, and then write a o3d.visualization.draw_geometries in pure Python, similar to what you did for jupyter_draw_geometries.
     • o3d.visualization.draw_geometries can take None and it calls o3d.visualization._internal_draw_geometries
3. Consider adding matplotlib as global default dependency in src/Python/requirements.txt, as many notebooks require it.
4. (can be done in future PR) Consider adding a build directory, such that the executed notebooks with outputs are stored in the build directory, where the entire directory is ignored by git. In this way.
   • building docs won't change git status,
   • and, we can potentially simplify the git logic in git_enable_ipynb_filter.sh
5. Looks like TriangleMesh.CreateMeshCoordinateFrame is failing the unit test
6. It may be useful to add visible parameter to DrawGeometries, and hide the visualizer window during jupyter execution
7. It maybe useful to convert the .sh files to .py files, so that the scripts can be run on windows
8. Regarding the broken examples, would it be OK to keep a list of them and exclude them in the current build? This PR is already quite big. I can help to address them together in a separate PR.

Reviewed 1 of 108 files at r6, 100 of 108 files at r7, 24 of 24 files at r8.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @griegler)

---

examples/Python/jupyter_run_all.sh, line 1 at r8 (raw file):

#!/bin/env bash

chmod +x this file and add to git
Or, shall we convert this to .py file, to enable this script on windows

---

examples/Python/jupyter_strip_output.sh, line 1 at r8 (raw file):

#!/bin/env bash

chmod +x this file and add to git
Or, shall we convert this to .py file, to enable this script on windows

---

src/Open3D/Visualization/Utility/DrawGeometry.cpp, line 53 at r8 (raw file):

                    bool mesh_show_back_face /* = false */) {
    Visualizer visualizer;
    if (visualizer.CreateVisualizerWindow(window_name, width, height, left,

it may be useful to add visible parameter here, and hide the visualizer window during jupyter execution

[griegler]

1. I changed the color_map_optimization.ipynb tutorial so that it automatically downloads the dataset if needed
2. Yes, it would be possible to change draw_geometries to a Python implementation, but this would break the compatibility with the C++ API?
3. Done
4. Can be done in another PR. However, I am not sure how practical this solution is. We can discuss this separately.
5. Fixed
6. Done. It is now coupled with the interactive flag
7. Done
8. Ok for me.

Reviewable status: 120 of 127 files reviewed, 4 unresolved discussions (waiting on @griegler and @yxlao)

---

examples/Python/jupyter_run_all.sh, line 1 at r8 (raw file):

Previously, yxlao (Yixing Lao) wrote…

chmod +x this file and add to git
Or, shall we convert this to .py file, to enable this script on windows

Done.

---

examples/Python/jupyter_strip_output.sh, line 1 at r8 (raw file):

Previously, yxlao (Yixing Lao) wrote…

chmod +x this file and add to git
Or, shall we convert this to .py file, to enable this script on windows

Done.

---

src/Open3D/Visualization/Utility/DrawGeometry.cpp, line 53 at r8 (raw file):

Previously, yxlao (Yixing Lao) wrote…

it may be useful to add visible parameter here, and hide the visualizer window during jupyter execution

Done.

[griegler]

@yxlao @germanros1987 @benjaminum
Travis fails, because it can not build the doc due to a GLFW error. This error is out of scope of this PR. Should the merge be blocking on the fix of the headless rendering, or are we good to merge?

[yxlao]

Or, can we skip those failing examples and keep its original docs? Later when those examples are fixed, we then create new ipynb docs.

Reviewed 9 of 9 files at r9.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @griegler and @yxlao)

[yxlao]

My previous comment is not optimal since most tutorials require visualization. Does this work https://docs.travis-ci.com/user/gui-and-headless-browsers/#using-services ?

Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @griegler and @yxlao)

[griegler]

@yxlao the last link looks useful. Would you mind to try it out?

[yxlao]

Sure, I can give it a try

[yxlao]

hi @griegler , you may check out my PR at griegler#3 and merge here to trigger the CI if appropriate.

---

nbsphinx always builds Jupyter notebooks if there are no outputs, and the build runs in parallel. This setup sometimes results in infinite hung (sees to be related to multiprocessing and pipe). We can execute the notebooks first and then call sphinx build. This way, we have more control over how the notebooks are executed.

Example commands:

python make_docs.py --clean_notebooks --execute_notebooks=auto --sphinx --doxygen

This PR also tries to enable xvfb in Travis. In my local machine, xvfb runs, but always gives black image while capturing screen buffer. If this is also the case for Travis, then we can only rely on Travis to test whether the notebooks can run, but we cannot determine the correctness of the outputs. Additionally, our docs server may be affected due to the xvfb setup.

[griegler]

@yxlao Running it with xvfb leads to a error in the jupyter notebooks for me: GLFW Error: X11: RandR gamma ramp support seems broken

[germanros1987]

Here is an overview of what got changed by this pull request:

Issues
======
+ Solved 1
- Added 6
           

Complexity increasing per file
==============================
- examples/Python/open3d_tutorial.py  8
         

Clones added
============
- examples/Python/open3d_tutorial.py  8
- examples/Python/jupyter_run_all.py  1
- examples/Python/jupyter_strip_output.py  1
- examples/Python/Misc/meshes.py  6
         

See the complete overview on Codacy