-
Notifications
You must be signed in to change notification settings - Fork 393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add notes on error-tolerant execution to CLI docs #972
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -26,18 +26,6 @@ jupytext --set-formats ipynb,py notebook.ipynb # Turn notebook.ipynb into a pai | |
jupytext --sync notebook.ipynb # Update whichever of notebook.ipynb/notebook.py is outdated | ||
``` | ||
|
||
For convenience, when creating a notebook from text you can execute it: | ||
```bash | ||
jupytext --set-kernel - notebook.md # create a YAML header with kernel metadata matching the current python executable | ||
jupytext --set-formats md:myst notebook.md # create a YAML header with an explicit jupytext format | ||
jupytext --to notebook --execute notebook.md # convert notebook.md to an .ipynb file and run it | ||
``` | ||
|
||
If you wanted to convert a collection of Markdown files to paired notebooks, and execute them in the current Python environment, you could run: | ||
```bash | ||
jupytext --set-formats ipynb,md --execute *.md | ||
``` | ||
|
||
You may also find useful to `--pipe` the text representation of a notebook into tools like `black`: | ||
```bash | ||
jupytext --sync --pipe black notebook.ipynb # read most recent version of notebook, reformat with black, save | ||
|
@@ -58,6 +46,37 @@ Note also that on Windows you need to use double quotes instead of single quotes | |
|
||
Execute `jupytext --help` to access the full documentation. | ||
|
||
### Execute notebook cells | ||
|
||
For convenience, when creating a notebook from text you can execute it: | ||
|
||
```bash | ||
jupytext --set-kernel - notebook.md # create a YAML header with kernel metadata matching the current python executable | ||
jupytext --set-formats md:myst notebook.md # create a YAML header with an explicit jupytext format | ||
jupytext --to notebook --execute notebook.md # convert notebook.md to an .ipynb file and run it | ||
``` | ||
|
||
If you wanted to convert a collection of Markdown files to paired notebooks, and execute them in the current Python environment, you could run: | ||
```bash | ||
jupytext --set-formats ipynb,md --execute *.md | ||
``` | ||
|
||
Please note: if any notebook cell errors, execution will terminate and `jupytext` will not save the notebook. This can cause headaches as the details of any error would be encoded in the notebook that isn't saved. But there's a way: `jupyter nbconvert` has a mode which will still save a notebook if a cell errors, producing something akin to what would happen if you ran all cells manually in Jupyter's notebook UI. | ||
|
||
```bash | ||
# First, convert script (py/sh/R/jl etc) -> notebook. May need additional args to define input format etc as above. | ||
jupytext --to ipynb script.sh | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like this but I am afraid most users convert There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll Also, yes, with bash_kernel (though it seems the latest release & HEAD are broken, see takluyver/bash_kernel#120) |
||
# Then, execute notebook in place and allowing cells to produce errors | ||
jupyter nbconvert --to ipynb --inplace --execute --allow-errors script.ipynb | ||
# One can also combine these to a single command using jupytext --pipe | ||
jupytext --to ipynb --pipe-fmt ipynb \ | ||
--pipe 'jupyter nbconvert --to ipynb --execute --allow-errors --stdin --stdout' \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Great example! |
||
script.sh | ||
``` | ||
|
||
In each of the above, `jupyter nbconvert` could be replaced with any alternative tool to execute a jupyter notebook non-interactively, including [papermill](https://github.com/nteract/papermill) which would allow notebook parameterisation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In my experience, Also would you mind to add a link to a post I authored a few years ago on Jupytext + Papermill ? It is available on medium ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. RE papermill: this is why
Though I really ought just make a PR there for Bash as it would be pretty simple. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As expected it turned out to be pretty easy. See nteract/papermill#674 |
||
|
||
|
||
## Notebook and cell metadata filters | ||
|
||
If you want to preserve (or filter out) certain notebook or cell metadata, change the value of either `notebook_metadata_filter` or `cell_metadata_filter` with the `--update-metadata` option. For instance, if you wish to convert an `.ipynb` document to a `.md` file and preserve all the notebook metadata in that document, run | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that having the notebook fail is an unpleasant experience, yet failing on error is also
nbconvert
's standard, so I'd rather introduce the next paragraph as something like advanced notebook execution. What do you think?