Add provisioner for new Runtime #2276
Conversation
…rintf-like formatting
STL is not very convenient if you want to produce a heavily-formatted string and `llvm::format` uses llvm streams to produce the output. The new `strFormat` function is useful if you need a convenient method to create a formatted `std::string`.
An example of `strFormat` use would be `std::string str = strformat("Happy %s Year %d", "New", 2019)`.
| /// Returns a ResultCode indicating if the operation was a success. | ||
| ResultCode | ||
| provision(std::vector<DAGNode> &networks, | ||
| std::map<DeviceIDTy, std::unique_ptr<DeviceManager>> &devices, |
There was a problem hiding this comment.
Minor nit: Is it important that devices are a sorted map?
There was a problem hiding this comment.
It doesn't strictly need to be, but the consistent ordering is useful for the round robin provisioning.
| Module &module); | ||
|
|
||
| private: | ||
| std::unique_ptr<Backend> backend_; |
There was a problem hiding this comment.
The provisioner owns the backend?
There was a problem hiding this comment.
Yes, since the compilation happens in the Provisioner.
Is there significant cost in initializing a backend? Currently the Provisioner will initialize the backend for each device. This could be optimized to check first, or have a map of backends for each device type.
| using namespace runtime; | ||
| using DeviceID = unsigned int; | ||
|
|
||
| void addNodes(std::queue<std::vector<DAGNode *>> &nextNodes, |
There was a problem hiding this comment.
I suggest giving a typename to "std::queue<std::vector<DAGNode *>>".
I am not sure why this is a queue. Isn't this the same as std::vector? Are you going to be processing this data structure inplace?
There was a problem hiding this comment.
It's being used as a work queue in the provision function, This stores the next set of functions to be compiled and added to a device. I don't think a separate function is still necessary though.
tests/unittests/ProvisionerTest.cpp
Outdated
| std::unique_ptr<DeviceManager> device(new CPUDeviceManager); | ||
| devices.emplace(i, std::move(device)); | ||
| } | ||
| auto result = provisioner.provision(networks, devices, *mod.get()); |
There was a problem hiding this comment.
Why are we moving the module to be owned by the provisioner? I think that it would make more sense for some external entity to own it and just pass a pointer. For example, our unit tests or ONNX-IFI.
There was a problem hiding this comment.
The provisioner doesn't own the module. The module is provided to the HostManager by ONNX-IFI as a pointer and that gets passed along to the provisioner. This unit test is making a custom module to test the provisioner handling multiple functions in one graph. The device map will be owned by the host manager, the provisioner takes in a reference.
* Create Error types * address comments
*Description*: Implement RowwiseQuantizedSparseLengthsWeightedSum. Added support for both Interpreter and CPU backends. *Testing*: Added unit tests for both RowwiseQuantizedSparseLengthsWeightedSum and RowwiseQuantizedSparseLengthsSum (implemented as the first but with weights as a float splat of 1.0). *Documentation*: Added to Quantization.md Related to pytorch#1698
|
ASAN is breaking because of the DAGNodes for the unit test, this will need to be updated after the Partitioner lands to match the new structure so I'll update and address the ASAN then. |
…t can be used outside of the EE. (pytorch#2289)
|
hmm looks like I did some git mistakes, I'll move this to a clean branch. I rebased locally but then did a pull instead of a force push.... This pr can be closed the new PR is #2293. |
Description: This PR adds the Provisioner for the new runtime. This is an initial provisioner so it's rather straight forward. Later work can be done to improve the provisioning algorithm. This is almost ready, it needs to be updated to the new DAGNode when it get's merged from #2268 .
Testing: Added new unittest which uses the CPU backend and CPUDeviceManager.
Documentation: N/A
Related to #2045