Modify FATE to allow distributed workers

[keith-turner]

FATE could be modified to allow distributed workers to run FATE transactions. Currently FATE relies on the manager lock to ensure one FATE instance is running. That single instance uses a in memory synchronized java map to reserve FATE transaction so that only a single thread runs. Could move to a model of using a condition update (in ZK or metadata table if #3559 is implemented) to reserve a FATE transaction, setting the process lock id per transaction. Would need to have a process of deleting locks held by dead processes.

Distributing FATE could allow scalability of operations like bulk import. It may also be an alternative to using a distributed work queue for the problem of split computation #3560 and compaction file selection #3526. This may be a prerequisite for #3247

[keith-turner]

#3823 and #3560 are related