Prepare for hybrid full chain with wc/ls stage separate from follow-on pure-WCT stage #131
Conversation
…torch. This is probably not the last on libtorch suport
Needed to directly load into the GPU. This is with libtorch 1.9.
…orch script node TorchService should use less memory and be thread safe but needs more testing
…on HDF5 to remove illegal cross dependency on hio
… the anode plane Note, this is a broken concept in general and will only work accidentally with some detectors.
There is a stand-alone PlaneSelector to do this and DNNROIFinding changes from cbeg/cend to using the same pattern internally. This is based on a channel selection from OmnibusSigProc that is moved into a new Aux/PlaneTools.h. But, OmnibusSigProc still is left as-is. It should be factored to use this moved code.
…ut scale is used in dnnroi
|
This is ready for others to look at. In particular, @HaiwangYu Here is one easy way to exercise things: The Jsonnet is in The Resulting PDF should show ADC, gauss and dnnroi frames with the simple cross-detector line source. I have some config in my frappe and toyzero packages that will read in depos dumped from larsoft, but I think this simple test gives the main gist. Here is the pgraph:
Note, this runs torch on the CPU with concurrency 1. To try |
|
Hi @brettviren, sorry for the delay. One thing I noticed first is that the gpvm UPS version of libtorch doesn't have cuda support. So maybe we need to add Otherwise, the compiling would fail with the gpvm libtorch I am still working on other testings and evaluations of this PR. |
|
Oh, good catch. Actually, based on our discussion, I meant to remove that cache purge which will also remove this explicit dependency. Since I've been developing on top of this branch for the upcoming |
For sure, that works too. |
There was a problem hiding this comment.
Hi @brettviren, I marked several small things. Most are just for myself.
I think we can merge it and fix them later.
Did some tests here:
https://www.phy.bnl.gov/~yuhw/wct-analysis/ml-sp/hybrid-wcls-test
|
|
||
| local app_plugins = { | ||
| 'TbbFlow': ["WireCellTbb"], | ||
| 'PGrapher': ["WireCellPgraph"], |
| @@ -5,17 +5,89 @@ | |||
| #ifndef WIRECELLPYTORCH_TSMODEL | |||
There was a problem hiding this comment.
Sorry for this inconsistent name of the include guard. Should change it next time.
|
|
||
| ## when building with __DEBUG__ defined in DNNROIFinding you need to | ||
| ## temporarily add HDF5 as a dependency. | ||
| # bld.smplpkg('WireCellPytorch', use='WireCellAux LIBTORCH HDF5') |
| { | ||
| std::lock_guard<std::mutex> guard(Hio::g_h5cpp_mutex); | ||
| h5::fd_t fd = h5::open(m_cfg["evalfile"].asString(), H5F_ACC_RDWR); | ||
| h5::fd_t fd = h5::open(m_cfg.debugevalfile, H5F_ACC_RDWR); |
| @@ -283,35 +287,25 @@ bool Pytorch::DNNROIFinding::operator()(const IFrame::pointer& inframe, IFrame:: | |||
| aname = String::format("/%d/frame_%s%d", m_save_count, "dlcharge", m_anode->ident()); | |||
| h5::write<float>(fd, aname, sp_charge.data(), h5::count{ncols, nrows}, h5::chunk{ncols, nrows} | h5::gzip{2}); | |||
There was a problem hiding this comment.
m_anode was removed. need to change to Factory::find_tn<IAnodePlane>(m_cfg.anode)
This starts by adding depo tar streams to gain better compression than what cnpy provides (none).
It also includes one small tweak to
DepoBaggerwhich will bag all depos regardless of their time if the time "gate" is [0,0]. For existing configs, this will not change behavior.Draft for now as I work out any more issues along the way to a full hybrid job.