Realtime printing

[eriknw]

Closes #13.

• Use events added to the latest distributed release to send print statements back to the client in realtime.
• We now always try to capture print statements.
• Also, we never block to display print statements or displaying the repr of the final expression (previously, we would block if not running in e.g. Jupyter Notebook).

Q1: is it safe to assume that events sent from a single task will be received in correct order (FIFO)? This appears to be the case.
Q2: should we also print on the worker?

CC @gjoseph92

[gjoseph92]

Thank you! Very excited about this.

Q1: is it safe to assume that events sent from a single task will be received in correct order (FIFO)? This appears to be the case.

I think so? That is, I don't think we're explicitly enforcing this, so it's probably theoretically possible to get out of order (unless we get some guarantees from TCP somehow), but seems very unlikely? @fjetter might know better since he added this feature.

Q2: should we also print on the worker?

Probably yes, since that's what distributed.print is doing right now. Could get a little weird running a local cluster in a non-Jupyter notebook (double prints), but why don't we go with yes for now.

[eriknw]

Cool, let's give this a try! Thanks for the input and pointing me to the distributed PR (dask/distributed#5217) that enables this.

I have another question: do you know of any alternatives to using ipywidgets.Output to asynchronously update cell outputs? I don't like how it requires an active kernel to display the outputs. Alternatively, is there some way to "persist" the Output object when we know we're finished updating it?

If out-of-order print statements become an issue, we can use the timestamps to ensure they're displayed in the correct order.

[eriknw]

I need to rework this. worker.log_event doesn't raise here--it raises an exception on the client instead. It also doesn't provide a return value that can be used to determine whether the log was successful or not.

I think the solution for afar is to not display the rich repr of objects that use _ipython_display_ or find some other workaround.

[fjetter]

doesn't provide a return value that can be used to determine whether the log was successful or not.

if that's helpful, we can change it

[fjetter]

Q1: is it safe to assume that events sent from a single task will be received in correct order (FIFO)? This appears to be the case.

Yes

Q2: should we also print on the worker?

no strong opinion ¯_(ツ)_/¯