Feature idea: a temporary execution barrier

[marius311]

Hi guys, over the last few days, after having toyed with Pluto for a while, I spent a solid chunk of time using it for my actual real work. Aside from continuing to love reactivity and wondering how I've gone so long without it, I think I pinpointed one feature which I feel like could take my productivity while using Pluto to the next level, which I thought I'd propose (and apologies if this has already been discussed somewhere).

For reference, I'm a physicist doing lots of free-form analysis work, probably pretty typical of your users in academia. Basically the thing I kept running into is I have a fairly "mature" notebook set up, and then I want to tweak something near the "beginning" of the DAG, but I might want to look at a few cells beyond that to check things are right, before letting the whole DAG run to the end, which may actually take minutes to tens of minutes in my case.

What I usually ended up doing is to just make the change, look at the "quick check" cells as the whole thing evaluates, and then just interrupt and try again if "things weren't right". Unfortunatey interrupt is not always reliable or even possible if external libraries are called, and somehow the whole thing felt frantic.

So my idea is basically just to add a way to place a barrier in the DAG either before or after a given cell (I ran into cases where both would be useful). I made a mock-up of what the UI might look like below. The button could live alongside the hoverable add cell above/below button, and once clicked would remain persistent (and can be clicked again to turn off). To visually indicate the temperary break in reactivity, the UI should dim all the subsequent cells in the DAG and hide their output. It would be something like this:

In terms of what is saved in the notebook file, maybe the break in the DAG can be indicated by just commenting out code which is beyond the barrier, maybe with a #= Pluto barrier ... =# or something to avoid confusing with user code.

I can imagine you might have some reservations about temporarily breaking reactivity, which I can definitely understand. I do think though that this isn't really getting rid of reactivity, instead its an optional "power" feature that kind of fits in the reactivity paradigm really well. If I recall too in your JuliaCon talk you mentioned work on a way to "check in" a bunch of changes at once. Maybe that's similar and you've already thought of this, it sounded like its trying to solve the same underlying problem, but if that's similar to "pausing" evaluation while editing several cells at once, I'll mention that the key thing here is that you get to look at the output of some cells after you've made a change.

Anyway, just an idea I had which I hope you guys can consider. I'm more than happy to discuss more or to be a guinea pig should you decide to purse something like this.

[fonsp]

Hi @marius311,

Thank you for the exciting suggestion! It sounds like you found this idea after some serious Pluto use, which makes it very valuable! I appreciate that you talked about the existing 'solutions' for this problem, and I can relate to your observations.

Your suggestion gave me an idea: you would have a "hotbar" in the corner of the screen with "filters" that you can drag onto your cells - this would be one of them. Other might be capturing terminal output, step-by-step debugging, memory profiling, a browser notification when the cell is done, saving cell output to a file, ... Other packages could declare their macros as Pluto filters by giving them an icon a colour - or something.

Even if it is just this single filter (execution barrier), I think that it would create a friendlier learning curve to hide it in a corner, instead of showing it next to every cell.

You talked about having the barrier both before and after a cell. I see why both would be useful, but the feature would be easier to understand if there was only one option - I would say after the cell. What do you think?

Another comment was that we don't really need to save it to a file, right? It seems more like something you want during your writing session.

And blurring the cells is a very smart idea! I am worried that breaking the reactivity can turn your mental model into chaos, but this might solve it.

A side note about stopping cells - of course it would be great if we can fix the problems that it has. The implementation is somewhat improvised, but I don't have any good notebooks to test it on. So I'd love to hear more about what you found, and maybe even an example of a notebook where Stop is broken? (#228)

Thanks again, look forward to hear from you.
-fons

[BrettKnoss]

I thought of barriers, but also execution blocks. I'm doing an accounting program, and if I could reload the dictionary where I have my accounts, and the transactions, then load the year end as a separate block, this would make things simpler. I think that if we could do notebooks in notebooks, and have variable blocks with shared packages and unique variables, and finally execution of a group of blocks, this would allow for complete documents. There is also a need for a sidebar navigation table, so if each block can show up in the nav bar with play buttons, and unique names, it would be a lot easier to go through more complex code.

[lungben]

For sidebar navigation, you can use TableOfContents() from PlutoUI.

Regarding calculation blocks, you could either put each block in one cell or use multiple cells defined such that they are dependent on each other, i.e. all run if the 1st one is triggered.

[marius311]

Thanks for the response! Some thoughts:

You talked about having the barrier both before and after a cell. I see why both would be useful, but the feature would be easier to understand if there was only one option - I would say after the cell. What do you think?

Yea I did worry too about the added complexity of before/after. The reason I wanted both is because I ran into both of these patterns:

1)
                     /--> heavy computation -\
input --> quick check --> heavy computation --> visualization, etc...
                     \--> heavy computation -/

2)
     /--> quick check 
input --> quick check
     \--> quick check
     \--> heavy computation --> visualization, etc...

In case 1 I wanted to put a barrier after quick check, and in case 2 I wanted to one before heavy computation. If you go with just a single barrier type, I think it should be the before kind though. If all you had was before, you could still do case 1 above, you just need to manually turn off each of the three heavy computation cells, which maybe isnt so bad. But if all you had was after, there would be no way to see the quick check cells without the heavy computation in 2.

Fwiw, maybe that makes the UI easier to reason about too since its like you enable the barrier on a cell and that very cell gets blurred (along with its dependents). Maybe if there's just a single button you'd consider putting it in the "gutter" next to the hide cell button? Tbh I'm less a fan of click and drag which feels clunky, that gutter just feels like the perfect place to stick a few buttons and I kind of figured you guys had basically always been planning that.

Another comment was that we don't really need to save it to a file, right? It seems more like something you want during your writing session.

I suppose its not critical, I agree its mostly during writing, I just thought it'd be nice to preserve the one-to-one mapping between a file and the current notebook state.

[oschulz]

I agree a "before"-barrier would be the most useful, I think it would also be more elegant conceptually.

[fonsp]

You're right that if there's only one, it should be the before type. This is similar to "commenting out" a block of code, except that you still see the output (half-faded), and dependent cells don't error. So it's a familiar concept, but 'reactified' - nice!

[j-fu]

Hi I have the feeling that such a feature could provide what I like to have in #259, provided that such a barrier also becomes an element of the file format or kind of can be optionally set for the first cell directly after loading. Moving the barrier could automatically trigger execution of the cells freed from it.

[ScottPJones]

For teaching and demos, what would be useful is maybe a button (like the present button, possibly using the present button), so that any cells that are past the current "slide" being presented are not yet evaluated.
When the notebook is loaded, it would be good to have some marker (stored in the notebook.jl file) to indicate how much of the notebook should be active. Given that it's a useful technique to put cells at the end, that don't need to be displayed, but need to be executed, without the cell info messing up the presentation, there would also need to be a marker to indicate where the end of the inactive region is.
Maybe this could be done in some general fashion, being able to indicate blocks of cells, which are inactive initially, and can be activated as a set (such as when the presentation reaches that slide, or a button on that "slide")

In my case, it is to have an interactive demo, showing off the advantages of the package being demoed as well as Julia - and incidentally, showing off the coolness of Pluto compared to Jupyter!

[fonsp]

@ScottPJones this mechanism uses the order of cells to determine the runtime - this is something that Pluto fundamentally should not do. The original suggestion uses the topological ordering of cells, which is why i think it could work within the paradigm

[Moelf]

@marius311 I love this idea, clearly you know the pain point of users of "our kind".

One comment: because of the reactivity model, we indeed have to do this "before" cell. Say we have a variable params that is used in heaby_func, you can't do "after the cell" without doing some cumbersome real_paras=paras in one cell and change another cell to heaby_func(real_paras) because we don't allow re-definition.

In summary how this should work:

1. mark a cell as barrier
2. do the usual thing except for each cell parallel and after the barrier cell in DAG, don't run them
3. instead grey out these cells's output.

[pbouffard]

If this is implemented it would be great if Pluto would automatically flag certain cells as long-running (after they have first run and been observed as such) and ask the user the next time they are about to be run if they want to defer execution (with a "no, and never ask me again" option of course). Bonus: Pluto could offer to wrap the cell with some kind of memoization.

[lungben]

Cells might only be long-running the first time e.g. due to compilation or Pkg operations. It would probably be quite tricky to automatically identify long-running cells without creating too much noise for the user.

[oschulz]

It would probably be quite tricky to automatically identify long-running cells

I also think that in many of the use cases in question, the user's knowledge will be needed. For example, one may often want to mark only the first cell in a linear dependency chain of cells doing heavy stuff as "no-auto-eval". Or too-frequent evaluation of cells may need to be prevented not because it takes long, but because it consume resources. For example, quite a lot of free web services (data APIs, etc.) limit query frequencies, or only allow a certain number of queries per month without paying for them, etc.

[compleathorseplayer]

I think the simplest for students is to base reactivity on wherever one runs a cell - those at or earlier would be reactive and those after would not be affected (I don't see any logical/topological inconsistencies in this and it is simple). The property of reactivity would not be permanent - just dependent on where one chose to execute. For full reactivity, one could merely always execute the last cell.

[fonsp]

If you change

x = 1

y = x + x

to

x = 2

y = x + x

then the second cell would be in an inconsistent state, right?

[compleathorseplayer]

If the second cell had not been executed then don't execute it. If it already had been executed, do not execute it again, but move 'forward' to the first cell treating the second cell like normal Julia code which had been previously executed. If there are more than 1 cell in the first section (above the click) require consistency in the first section (only).
I think this is logically consistent.

Perhaps someone else's suggestion about tick boxes for which cells you want to be reactive would be better, and it would work in a similar way - where if it a 'non-reactive' cell had been executed previously, it would be treated as past executed Julia code as normal, and only newly executed cells would be required to be consistent with each other.
[Those not marked as 'reactive' and not previously executed would be ignored]

[fonsp]

Is your suggestion the behavior of Pluto.run(run_notebook_on_load=false)? (try it out) I don't understand what you mean, it sounds too complicated to be a useful default

[gszep]

Hi everyone! So I'm using Pluto together with JSServe and Makie to create educational machine learning content and interactive analysis pipelines for experimental biologists and would definitely benefit from such a feature. I started discussing this here #803.

Suppose I want to create a book on machine learning, (imagine trying to create an interactive version of this book). A collection of cells can be considered a section or subsection. A single notebook can be considered a chapter.

The overarching goal is to allow the reader to easily navigate across chapters, interact with plots that have live demos of how certain algorithms work, and allow them add their own custom code to the algorithms to see how their modification/extension changes the results.
This must be done whilst minimising load time and time to first plot.

The way I see this working:

• The user loads the book from a sysimage that contains all the compiled functions and modules. Once loaded, this sysimage is shared across multiple notebooks files, each representing a chapter of the book - kind of like how you would organise a book built in LaTeX. Something like this Pluto.run( sysimage="book_image.so", notebooks=["chapter1.jl","chapter2.jl" ... ])
• On run a table of contents with chapter names and links appears, but none of the chapters have actually been loaded, only the sysimage is loaded. This table of contents should be accessible even when the reader has navigated into a particular chapter, kind of like PDF readers.
• Within each notebook/chapter there may be multiple sections that contain interactive demonstrations for different algorithms. We don't want them to all load at once. Ideally we would only load the section that the user would like to interact with / has navigated to. The problem is that these sections would likely want to share the value of Observable types.

I like the idea of blurring out sections and when the user clicks on a section it begins to load. As soon as an observable in one section is changed, rather than immediately passing the updated value to sections that listen to this observable, those sections are simply blurred out, signalling to user that changes they made require re-computation.

[fonsp]

Those are cool ideas! But most of this is off-topic, can you remove your comment here and start a new discussion?

[fonsp]

For example, you mention sectioning your notebook, this is being discussed here: #297

[oschulz]

I think being able to mark individual (typically compute heavy) cells as "don't auto-update" would open up Pluto to a whole new range of applications. While the user is playing around with some cells that (e.g.) define the input of a calculation, they typically will not want their (maybe chain of) expensive "main calculation" cells to be re-run all the time.

Another aspect, beyond usability of a notebook with compute-heavy cells, is resource usage: In contexts where cluster or cloud job submissions are involved, running certain cells might actually consume limited computation budgets (or money, in the case of cloud). Even if not, it might load a system heavily, affecting other users. So allowing for optional control of the update-mechanism would also be resource-friendly.

I like the idea of "blurring" or "greying out" outdated output of such cells (and the output of cells than depend on them). I think it should be done in a way that keeps the output readable (people may still want to see an outdated plot or so) but that is clear enough to give the intuition that it's not up to date.

[oschulz]

I think this also relates to #959 (Use Pluto to control physical systems). I definitely have my eye on Pluto as a possible graphical interface to control things and run/montitor measurements in our lab (which runs mostly on Julia software by now). But I would definitely want a barrier before cells that contain code like "ramp up the high voltage now", "start the vacuum pump", etc. ... :-)

[lungben]

Agreed, this would be very helpful. Together with #891 this should mitigate the disadvantages of reactivity without loosing its advantages.

@oschulz Btw. Connor is working on Pluto notebooks as Rest APIs, this may also be interesting for you.

[oschulz]

Btw. Connor is working on Pluto notebooks as Rest APIs, this may also be interesting for you.

Oh, indeed, that could be very interesting.

[fonsp]

I never thought of this aspect, thanks for bringing it up, and I would definitely like to hear more about your experience with this problem!

[oschulz]

I'll try to dial in to PlutoCon (not sure if I can make it yet).

[lungben]

Summarizing the discussion above / my suggestions for implementation:

Use Cases:

1. give a method to prevent auto-update of cells which take too long to compute to keep the notebook reactive and/or save computational resources.
2. step-by-step computations for educational purposes

Suggestion:

1. Allow the users to add execution barriers to Pluto cells from inside Pluto.
2. These barriers block execution of the cell they are added to (i.e. located before the cell) if active.
3. The barriers should be stored in the notebook.jl file, e.g. as comments.
4. If the notebook.jl file is executed in Julia (without Pluto), the whole file should run, ignoring execution barriers (there would be no way to turn them off without Pluto anyhow).
5. When opening a notebook.jl file in Pluto, all barriers should be active by default, so that file opening is quick.
6. In Pluto, you should be able to activate, deactivate or delete execution barriers.
7. If an execution barrier is active, the cell and all cells depending on this cell should be marked gray in Pluto, so that it is obvious that they are not executed.
8. When deactivating an execution barrier, the corresponding cell and all dependent cells are executed immediately and not grayed out anymore.

Technical implementation:

1. Execution barrier information (is there a barrier? is it active?) could be added to the Cell structs in the backend.
2. Based on which cells are deactivated and the cell dependency information, the backend decides which cells to evaluate in case of changes.
3. In the frontend, the information whether a cell is grayed out is determined based on the execution barrier information in the Cells and the cell dependency information from Visualize cell dependencies (Backend) #891
4. Frontend functionality for adding, deleting, activating and deactivating of execution barriers plus visualization of execution barrirers required.

Happy to take a first look at the backend!

[lungben]

Started with implementation: #985

[j-fu]

Hi, just would like to remind about #421 in this context. We would need to store more cell metadata to implement what has been proposed. Would love to try a PR but not before April...

[lungben]

I would keep these topics separated. For now, information about execution barrier could be stored as a comment in the 1st line of the code.

[j-fu]

Sure, this is reasonable, as things seem to intersect only at this point.

[fonsp]

I like your suggestion, but having a backwards-forwards compatible file format is most important.

[fonsp]

implemented in #985

[oschulz]

Awesome!

Quick question: Currently, the cell has to be manually enabled and then disabled again for the "long-runnning" computations scenario, correct? Would it be possible to add a "one-shot-enable" capability, maybe even with a keyboard shortcut?

[lungben]

I think this would be helpful, see #1209 (comment)

However, due to my lack of JS knowledge someone else needs to implement it. Any volunteers?