Rethinking architecture of jobs

[heavelock]

In result of #109 I have a bit of a problem and few thoughts about the architecture of jobs that are used here.
I was calculating MWCS for few stack types for different filters. calculate_mwcs is performing calculation on basis of the DTT jobs and checking them for having a flag T. First it looks takes one filter type, afterwards a components pair and performs calculations for all stacks. After performing all calculations for a job, it marks it as a D.
In my case, since this bug occurred in beginning of calculations for 4th filter(out of 5 prepared) I'm stuck with 2/5 of calculations not performed with all jobs marked as I which MSNoise doesn't interpret as something that it should take care about. To force MSNoise to continue calculations I had to:

1. Deactivate all filters for which calculations were done properly.
2. Deactivate a filter that caused exception
3. Change a status of all jobs to T.
4. Run once again msnoise compute_mwcs

My proposition is to instead of performing a MWCS as stack of for loops iterating over all possible parameters, modify a bit a whole job dispatcher module. Or rather create one :)

My proposition of solving that problem:
Simply add a new table in database which would be called MWCS Jobs (or similar :)) which would contain full information about each single MWCS job. The MWCS worker could in that case fetch a jobs to do from that table and just mark it after finishing calculations.
Changing the architecture to follow that would make calculations way more error resistant and would make using that module way easier. What's more, applying multiprocessing would be easy as pie since we could in the beginning fetch jobs to do from the database and create iterator of parameters that would allow to run as many of processes as user wants.

What do you think?