Create a new run pipeline interface

[jamesnakagawa]

• Encompass both single and batch runs in 1 ix
• Search for data sets and be able to restrict by CDTs present in selected pipeline
• Add from search results to specific inputs (add to buckets)
• Rearrange order of datasets for each input
• Customize combinations of datasets for each run

[jamesnakagawa]

Two interface possibilities conceived of with @donkirkby and @rhliang

Note that in these examples, when data sets and inputs are 1:1, it's a single run. Greater ratios mean batch runs.

This interface has the advantage of displaying full details of a CDT, but does not allow the user to rearrange/manipulate dataset combinations on the same page.

This interface has the advantage of allowing the user to control everything on one page. Full details are displayed in the search results box, but not the run table.

Accumulator controls are duplicated for each pipeline input. On Friday I thought this would become too visually complex, however, I realized today only the single "add" control is necessary to duplicate. "Remove" can be shown contextually when data sets are selected, and "add all"/"remove all" are not necessary when multiple-selection is a feature.

One potentially complex area here is the combination of drag+drop controls (for manipulating run combinations) and of select+direct controls (for adding/removing data sets). I think that some of the select+direct controls could be translated to a drag+drop schema but the converse would be awkward. Even then, it may be a rather complex system to navigate and some helper text could be useful here.

Of these two options, the second one is growing on me more. While there are still concerns about the complexity I think the challenge can be met. One suggestion on Friday was to have both interfaces as options (depending on user preference), although, that too adds complexity.

[jamesnakagawa]

I've taken the second option a step further:

After selecting a pipeline to run, the user is presented with this page:

The intuitive action is to click one of the "+" buttons to add a dataset to the input. On clicking, a slide-down animation will reveal a search dialog above the initial graphic:

The user can select one or more data-set on this part. Once the arrow button is pressed, though, the dialog shifts focus to the next input (and a "+" appears below the added dataset). This automatic next-step sequence continues until all inputs have ≥1 data sets. However, the user can circumvent this guided sequence and manipulate datasets arbitrarily (add out of order, delete, rearrange) by ignoring/closing the search dialog.

[donkirkby]

Sounds good, @jnak-cfe. If it's not too much work, try to save the state of each search form when it disappears. If the user clicks on one of the plus signs without actually adding any datasets, it would be nice to go back to the first search form and still see all the results there.

[jamesnakagawa]

I agree, good idea @donkirkby

[rhliang]

Looking good! I think the slide down is a great idea especially; keeps the focus at the top but doesn't disrupt the workflow, if that makes sense.

One question: does it make sense to automatically move to the next input when the arrow is pressed? I could envision situations where you have to do a couple of search queries to find all of the data you're looking for before moving on to the next column.

[jamesnakagawa]

@rhliang That's still fairly easy to do, it's just not the "guided" order of events. My thinking was that the most common use-case would be a single run, so the highlighted buttons "guide" the user through this task by default.

[donkirkby]

@rhliang, what would you think of using a Ctrl+click to add datasets without advancing to the next input search? A plain click advances to the next one, as James described.

[rhliang]

Yeah I think that could make sense, akin to how a chat client uses a shift-return to insert newlines. Maybe shift-click instead of ctrl-click? I feel like ctrl-click often means "select multiple".

[jamesnakagawa]

Definitely a power-user feature, but I like it.

[jamesnakagawa]

As discussed in meeting, this is in-progress in a nonmaster branch.

Next steps at the current moment:

• Set up run_pipeline to accept the plural run data (help please!)
• Create an API call which fetches datasets according to date last used (help please!)
• Create an interface for adding a dataset to span an entire column across runs
• Allow users to manipulate the filled-in run table with drag & drop

[jamesnakagawa]

Context: One of the next features of this interface is the ability to select 1 dataset to span all runs for a single output. We explored several options for this during the meeting yesterday, with consideration going to the burden of visual clutter, the understandability of any exposed controls, and the needed functionality.

We talked about putting auto-fill suggestions inside blank cells (or outside of them), putting a numeric input as part of the search dialog which could specify how many cells to fill, or splitting the green "insert" button into several different "insert" options.

So far, my favoured solution is a variation on the last one. There could be a "options menu" button within the "insert" button to expose a number of alternatives.

• The menu could include icons, or no icons (as icons are not guaranteed to be very helpful for such a specific operation)
• The first option would fill the column but any new rows (runs) would start blank.
• The second option would fill the column and any new rows would already be filled in. If you wanted to stop this behaviour, I guess manually deleting one entry should turn this "off", unless we a dedicated interface control for turning it off (but I don't think that's necessary)
• Maybe have a third option, "Fill only blank cells in column"?

[jamesnakagawa]

@rhliang and I met to discuss where this could go next.
I'm splitting this issue into #536, #537, #538, and #539.