[graph] Add clone() method for Module, copying all non-placeholder nodes

[nickgg]

Description: Add ability to clone a Module, which creates a copy of the graph and all constants but that shares Placeholders and Types.
** For Constants: we copy the backing Tensor memory, since compiling Functions in a Module can be destructive of the Constant representation.
** For Placeholders we now have multiple Modules holding a raw ptr, and need a count to manage lifetimes. The obvious solution is to use a shared_ptr here but that would change a lot of interfaces and I wasn't sure about it. Instead I've added a Refcounted trait which has a similar effect. I think we should work out the right ownership model here sometime soon.
** For Types we can just keep a shared_ptr to the list since it is not modified after graph create time. There is no thread safety across cloned Modules, just as there is no thread safety when using a single Module.
Testing: ninja test with the new test verifying cloned modules share only Placeholder objects in the graph.
Documentation: n/a

[beicy]

Thanks for working on this clone! This feature may be helpful for graph partitioning :) I would like to understand why in your case the placeholders could be shared by modules. Does that mean all the modules must use the same input?

[nickgg]

@beicy I believe the placeholder's must be shared with the clone if we want to drive it with the same kind of Context. All cloned modules must use the same input name and types/shape but the actual input & output tensors live in the Context object and each module can be driven by distinct contexts.

So we can do something like this:

Module mod;
Function *F = mod.createFunction("main");
Caffe2ModelLoader loader(..., ..., {"input"}, inputType, *F);
Placeholder* input = loader.getNodeValueByName("input");
Placeholder* output = loader.getSingleOutput();

auto modClone = mod.clone();

auto compiledFunc1 = backend_->compile(mod.getFunction("main"));
auto compiledFunc2 = backend_->compile(modClone->getFunction("main"));

Context c1, c2;
c1.allocate(mod.getPlaceholders());
c2.allocate(mod.getPlaceholders());

updateInputPlaceholders(c1, {input}, {inputTensor1});
updateInputPlaceholders(c2, {input}, {inputTensor2});

compiledFunc1->beforeRun(c1);
compiledFunc2->beforeRun(c2);

compiledFunc1->execute();
compiledFunc2->execute();

compiledFunc1->afterRun(c1);
compiledFunc2->afterRun(c2);

Tensor &output1 = c1.get(output);
Tensor &output2 = c2.get(output);

And output1 and output2 will contain the output of their respective inputs, even if the execution happened in parallel.

[nickgg]

The next step will be to avoid the double compile and clone the CompiledFunction.

[beicy]

@nickgg Thanks! I misunderstood the "shared placeholders" -- I thought they shared the same memory location. I think it will be good to add your above example to unittests:)

[nadavrot]

@nickgg Why do we need to clone the module? I thought that we just want to clone functions within the module No?

[nickgg]

@nadavrot I think cloning the module is useful for two reasons:

1. The current DeviceManager interface (see [glow/runtime] First cut DeviceManager with CPU backend #2134) uses std::unique_ptr as its interface, meaning if we want to put the same Module (or Function) on multiple devices we need to copy it. The reason for this is:
2. Optimize and compile of Functions is destructive to the Module, both by transforming the Function graph and also the format of the Constant Tensors. You cannot safely compile the same Function twice, even using the same backend. Due to Constant modification I'm not totally confident you could compile a Function and it's clone safely in the same Module.

[jfix71]

@nickgg RE: # 2, our optimizations should create a copy of the Constant being modified (e.g. see getUniquelyUsedConstant() in GraphOptimizer.cpp). So there should be no issue cloning a function and compiling both functions. In fact, if both functions will perform the same optimization to the Constant, then after you compile each of them the optimized Constant will be duplicated. We would need constant deduplication to recombine them, or do something more intelligent here to avoid requiring constant deduplication.

[bertmaher]

The current DeviceManager interface (see #2134) uses std::unique_ptr as its interface, meaning if we want to put the same Module (or Function) on multiple devices we need to copy it.

I'm concerned by needing to recompile a function just to run it on a different device. Is there a fundamental reason why that is necessary?

[nickgg]

@bertmaher No, theres not - it just seemed like an easier intermediate step for prototyping. I think since our designs have moved past this and this seems like not a safe or smart thing to do I'll just abandon this and we can come back to it if we need it.