1596 Extend lb stats #1601

lifflander · 2021-11-04T16:39:06Z

github-actions · 2021-11-04T16:43:19Z

PR tests (gcc-9, ubuntu, mpich, zoltan)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T16:50:41Z

PR tests (clang-3.9, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T16:51:20Z

PR tests (clang-5.0, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T16:53:25Z

PR tests (gcc-6, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T16:54:38Z

PR tests (gcc-5, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T16:55:14Z

PR tests (gcc-10, ubuntu, openmpi, no LB)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T17:01:02Z

PR tests (clang-9, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T17:04:28Z

PR tests (nvidia cuda 11.0, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T17:09:54Z

PR tests (gcc-8, ubuntu, mpich, address sanitizer)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T17:46:21Z

PR tests (gcc-7, ubuntu, mpich, trace runtime, LB)

Build for b74642a

FAILED: src/CMakeFiles/vt.dir/Unity/unity_10_cxx.cxx.o 
/usr/bin/ccache /usr/bin/g++ -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DVT_NO_COLOR_ENABLED -I/vt/lib/CLI -Irelease -I/vt/src -I/vt/lib/json/include -I/vt/lib/brotli/c/include -I/vt/lib/libfort/lib -isystem /vt/lib/fmt/include -isystem /build/checkpoint/install/include -isystem /build/detector/install/include -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -O3 -DNDEBUG -fdiagnostics-color=always -O0 -g --coverage -fPIC -std=c++14 -MD -MT src/CMakeFiles/vt.dir/Unity/unity_10_cxx.cxx.o -MF src/CMakeFiles/vt.dir/Unity/unity_10_cxx.cxx.o.d -o src/CMakeFiles/vt.dir/Unity/unity_10_cxx.cxx.o -c src/CMakeFiles/vt.dir/Unity/unity_10_cxx.cxx
In file included from src/CMakeFiles/vt.dir/Unity/unity_10_cxx.cxx:10:0:
/vt/src/vt/vrt/collection/balance/stats_data.cc: In member function 'std::unique_ptr<nlohmann::basic_json<> > vt::vrt::collection::balance::StatsData::toJson(vt::PhaseType) const':
/vt/src/vt/vrt/collection/balance/stats_data.cc:67:44: error: 'using ElementIDStruct = struct vt::elm::ElementIDStruct {aka struct vt::elm::ElementIDStruct}' has no member named 'home_node'; did you mean 'curr_node'?
       j["tasks"][i]["entity"]["home"] = id.home_node;
                                            ^~~~~~~~~
                                            curr_node
/vt/src/vt/vrt/collection/balance/stats_data.cc:68:50: error: 'using ElementIDStruct = struct vt::elm::ElementIDStruct {aka struct vt::elm::ElementIDStruct}' has no member named 'migratable'; did you mean 'isMigratable'?
       j["tasks"][i]["entity"]["migratable"] = id.migratable;
                                                  ^~~~~~~~~~
                                                  isMigratable
/vt/src/vt/vrt/collection/balance/stats_data.cc:116:64: error: 'struct vt::elm::ElementIDStruct' has no member named 'home_node'; did you mean 'curr_node'?
         j["communications"][i]["from"]["home"] = key.fromObj().home_node;
                                                                ^~~~~~~~~
                                                                curr_node
/vt/src/vt/vrt/collection/balance/stats_data.cc:119:60: error: 'struct vt::elm::ElementIDStruct' has no member named 'home_node'; did you mean 'curr_node'?
         j["communications"][i]["to"]["home"] = key.toObj().home_node;
                                                            ^~~~~~~~~
                                                            curr_node
/vt/src/vt/vrt/collection/balance/stats_data.cc:134:60: error: 'struct vt::elm::ElementIDStruct' has no member named 'home_node'; did you mean 'curr_node'?
         j["communications"][i]["to"]["home"] = key.toObj().home_node;
                                                            ^~~~~~~~~
                                                            curr_node
/vt/src/vt/vrt/collection/balance/stats_data.cc:149:64: error: 'struct vt::elm::ElementIDStruct' has no member named 'home_node'; did you mean 'curr_node'?
         j["communications"][i]["from"]["home"] = key.fromObj().home_node;
                                                                ^~~~~~~~~
                                                                curr_node
/vt/src/vt/vrt/collection/balance/stats_data.cc: In constructor 'vt::vrt::collection::balance::StatsData::StatsData(const json&)':
/vt/src/vt/vrt/collection/balance/stats_data.cc:200:70: error: too many initializers for 'vt::vrt::collection::balance::ElementIDStruct {aka vt::elm::ElementIDStruct}'
             auto elm = ElementIDStruct{object, home, node, migratable};
                                                                      ^

Build log

github-actions · 2021-11-04T17:47:04Z

PR tests (nvidia cuda 10.1, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

github-actions · 2021-11-04T17:49:25Z

PR tests (clang-10, ubuntu, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

codecov · 2021-11-05T00:38:38Z

Codecov Report

Merging #1601 (f2b4a55) into develop (c10ce38) will increase coverage by 0.02%.
The diff coverage is 82.27%.

❗ Current head f2b4a55 differs from pull request most recent head 832480a. Consider uploading reports for the commit 832480a to get more accurate results

@@             Coverage Diff             @@
##           develop    #1601      +/-   ##
===========================================
+ Coverage    83.05%   83.08%   +0.02%     
===========================================
  Files          786      785       -1     
  Lines        29795    29824      +29     
===========================================
+ Hits         24747    24779      +32     
+ Misses        5048     5045       -3

Impacted Files	Coverage Δ
src/vt/context/runnable_context/lb_stats.impl.h	`100.00% <ø> (ø)`
src/vt/messaging/active.h	`73.43% <ø> (ø)`
src/vt/objgroup/manager.h	`100.00% <ø> (ø)`
src/vt/termination/interval/interval.h	`90.00% <ø> (ø)`
src/vt/vrt/collection/balance/baselb/baselb.h	`100.00% <ø> (ø)`
src/vt/vrt/collection/balance/lb_common.h	`100.00% <ø> (ø)`
...c/vt/vrt/collection/balance/lb_invoke/lb_manager.h	`100.00% <ø> (ø)`
src/vt/vrt/collection/balance/model/load_model.h	`100.00% <ø> (ø)`
src/vt/vrt/collection/balance/model/raw_data.h	`100.00% <ø> (ø)`
src/vt/vrt/collection/balance/node_stats.h	`100.00% <ø> (ø)`
... and 76 more

github-actions · 2021-11-05T00:53:46Z

PR tests (clang-10, alpine, mpich)

Build for 832480a

Compilation - successful

Testing - passed

Build log

src/vt/elm/elm_id.h

PhilMiller · 2021-11-06T23:35:18Z

src/vt/elm/elm_id.h

@@ -77,6 +77,7 @@ struct ElementIDStruct {
  ElementIDType id = no_element_id; /**< id must be unique across nodes */
  NodeType home_node = uninitialized_destination; /**< the home node */
  NodeType curr_node = uninitialized_destination; /**< the current node */
+  bool migratable = false; /**< whether entity is migratable */


Maybe not necessary right now, but this seems like something in which clever representation seems worthwhile - e.g. if there's a no_node sentinel value (uninitialized_destination?), then setting curr_node = no_node should be suitable for a predicate member bool isMigratable()

src/vt/objgroup/holder/holder_base.h

src/vt/objgroup/manager.cc

src/vt/vrt/collection/balance/temperedlb/temperedlb.cc

src/vt/vrt/collection/manager.cc

PhilMiller · 2021-11-06T23:47:11Z

This all looks good to me. ZoltanLB needs to be updated. With that, I'd have no objection to merging this as is, and rolling forward to another PR

src/vt/elm/elm_id.h

src/vt/messaging/active.h

src/vt/objgroup/manager.impl.h

lifflander requested a review from PhilMiller November 4, 2021 16:41