[Intention] Remove image rendering from indices and concatenate indices analysis

[atruskie]

Although we build and maintain a monolith, our AnalysisPrograms.exe application has the concept of actions. Each action acts like its own program and allows us to do a broad range of analysis for different types of studies. Packaging our application as a monolith is useful for maintenance and we won't be changing that.

Expected behaviour

Each action of our program should more or less do one thing. For example, we can generate indices, render a visualization, or run an event recognizer. This follows the Unix philosophy of creating programs (in our case, sub-programs/actions) that do one thing which allows users to compose the results as they like.

Actual behaviour

Two of our actions (indices generation and indices concatenation) are very complicated because they render a visualization. Generally, the default visualizations they render are only appropriate for some of the analysis input/configuration options that are available. The results are

• visualization code that sometimes works or sometimes doesn't, depending on the analysis that was run
• multiple code paths create a heavy maintenance burden.
• complex command line and configuration switches that are unrelated to the main analysis, and are duplicate functionality of existing visualization actions
• in the case of concatenate indices multiple (redundant) executions of index concatenation have to be done to render different visualizations (See recently introduced features from Concat gap rendering #138 )

Any other details

This change is a low priority enhancement that may not be implemented for a while. We encourage any interested stakeholders to comment on the issue.

[towsey]

Before you implement this, we need to talk. I think there may unintended consequences here.

[atruskie]

Okay, We've since reversed our stance on this. The indices produced by the current concat command are unreliable.

We'd prefer people did quantitative analysis on raw indices from each source audio files, and used concat to produce images. This will reflect the defaults going forward.

[towsey]

Sorry to reopen this but I want to understand your logic.
The idea of "one command does one and only one function" makes things clean but it means every function must produce an output. In particular it means that concat would have to produce concatenated csv files. But you say these are unreliable. Why are they unreliable - is it because of timing issues i.e. recordings do not start on a minute?

One command - one function would work but then need to offer powershell scripts to tie workflows together.

[atruskie]

In particular it means that concat would have to produce concatenated csv files.

Concatenate's primary utility thus far has been rendering images. I'm suggesting it should only do that.

Why are they unreliable - is it because of timing issues i.e. recordings do not start on a minute?

Yes, that's exactly why. With our current code, we produce images and indices that are not aligned to minutes of a day and can pad on every join. We can produce in some scenarios 1460 minutes per day! And that's an average case scenario, not worse case.

One command - one function would work but then need to offer PowerShell scripts to tie workflows together.

Absolutely not.