nbformat 5.0?

[dhirschfeld]

I have wondered for a while if the notebook format wouldn't be better off as a directory of files/subdirs with associated json metadata - pretty much how it's done here. It would then be up to the front-end to combine them into the notebook UI the users interact with.

It seems this would alleviate problems integrating with version control and might make other things such as debugging easier: jupyter/jupyter_client#446

This isn't really an issue per-se (so feel free to close) - I'm just curious if this is something being thought about for a future version of nbformat?

They are more unwieldy in many ways, though

...or if the problems it creates are worse than those it solves?

[takluyver]

This particular design is unwieldy because each cell becomes a separate file, and they're not necessarily in order when you diff or merge. Of course, you can hook in tools to piece them together, but if you need special diff/merge tools, it's not a big advantage over JSON notebooks.

I kind of think that git and filesystems are missing a collection like a list. A directory is a key-value mapping like a dict, but there's no way that I know to have a sequence of nameless files. So this project clumsily emulates it with the 'sequence' files. I don't think there's any technical reason that filesystems couldn't have a 'sequence directory', but there's decades of precedent for what a filesystem is, and this is not part of it.

So the best we can do for version control is probably still something based around a text file for the entire notebook.

There are a bunch of other projects experimenting with variants on this idea, including jupytext (the current hot new thing) and ipymd. I also have another experiment nbhtml - the idea is that you could pair a source-only notebook in version control with a 'complete' notebook including outputs.

My guess is that adopting a radically different format as the new official notebook format would be a gargantuan task now: so many projects affected, so many people with different sets of priorities. But I'm no longer closely involved in Jupyter core development, so I don't know what the latest thinking is.

[dhirschfeld]

Thanks for the background! I still like the concept, but I guess there's not much support for such a radical change without someone very actively driving it (and demonstrating the benefits).

I'll go ahead and close as answered...