Common processing loop

[mottosso]

Goal

Provide a common interface for graphical user interfaces as well as custom and/or complex processing loops.

Overview

(1) pyblish.plugin provides (internal) low-level functionality towards Pyblish. (2) pyblish.util on the other hand offers a (public) high-level interface for common tasks. (3) pyblish-qml and pyblish-lite then provides the artist interface in the form of graphical user interfaces.

What's missing here is (4) low-level, but public interface for developers to build their own complex tools and solve complex requirements.

It should be simple enough for general use, yet complex enough to accommodate for a fully featured graphical user interface.

Motivation

pyblish.util, pyblish-qml and pyblish-lite currently maintain their own unique processing loops. The benefit of this has been that changes in one loop does not negatively affect another which has made it welcoming to experiment with new approaches to processing in search of the most optimal one.

A disadvantage to such an approach however is that maintenance is multiplied by the number of implementations. Tests are implemented multiple times, and odds are one behaves slightly different then the next.

This issue is an attempt to solve this.

Demotivation

I've tried this before.

pyblish.logic.process (now deprecated) provided an attempt at bridging the gaps between GUI, CLI and Python interface. The problem was subtle; too complex for general use yet too simple for the requirements of any GUI.

Implementation

engine.py will be in the family of util.py; that is, a completely standalone entity with no other dependencies than pyblish itself, and pyblish-base not depending on anything within it. A top-level dependency. util.py may eventually transition to using this.

The recent success in implementing the processing loop for pyblish-lite spawned the idea that perhaps such a loop could be made available on a more general basis and become shared across all interfaces.

An implementation must enable:

1. Scripting
2. Command-line use
3. Graphical user interfaces

And as such must be:

1. End-user friendly
2. Asynchronous

End-user friendly

At the end of the day, I'd like to enable the end-user with the power to implement their own complex requirements using engine.py. For example, bespoke graphical user interfaces, such as the one developed at CAOZ, or unique one-off tasks, such as wedges in Maya at Colorbleed.

Asynchronous

It must be asynchronous, a GUI should enable use even though it may be busy processing. This is because processing is expected to take anywhere between a few microseconds to minutes.

Because it is asynchronous, it must also be stateful. As in, no function provides anything via return value. Instead, values are provided via signals and slots.

engine = pyblish.engine.create_default()

def on_collected():
  context = engine.context
  context.data["myValue"] = 5

engine.was_collected.connect(on_collected)
engine.collect()

For true asynchronicity within a Qt application, we must leverage the event loop and as such provide hooks for what mechanism actually instantiates signals. Since command-line interfaces and general scripting cannot require Qt, there must also be an alternative, non-blocking mechanism of signals to be used where no better replacement exists.

from Qt import QtCore

def defer(delay, func):
  return QtCore.QTimer.singleShot(delay, func)

engine = pyblish.engine.create(
  signal=QtCore.Signal,
  base=QtCore.QObject,
  defer=defer)

In this case, signals throughout this engine are Qt signals. Because Qt signals also require the baseclass to be a QObject, the existing superclass is replaced dynamically. Asynchronous calls are made with defer. This is what allows the GUI to process graphics inbetween making calls to blocking functions, such as the process() on a plug-in.