Improve MG graph creation #2044
Conversation
…e and also seems like having an issue with 2^31 or more elements)
seunghwak
added
2 - In Progress
improvement
seunghwak
changed the title
seunghwak
added
3 - Ready for Review
bug
Codecov Report
@@ Coverage Diff @@
## branch-22.04 #2044 +/- ##
===============================================
Coverage ? 73.55%
===============================================
Files ? 156
Lines ? 10295
Branches ? 0
===============================================
Hits ? 7573
Misses ? 2722
Partials ? 0 Continue to review full report at Codecov.
|
| auto lower_it = | ||
| thrust::lower_bound(thrust::seq, group_id_first, group_id_last, static_cast<int>(i)); | ||
| auto upper_it = thrust::upper_bound(thrust::seq, lower_it, group_id_last, static_cast<int>(i)); | ||
| return thrust::make_tuple(static_cast<int>(i), | ||
| static_cast<size_t>(thrust::distance(lower_it, upper_it))); |
There was a problem hiding this comment.
Instead of casting i to int can you cast it to
typename std::iterator_traits<GroupIdIterator>::value_type while you are using it for binary_search?
It could be cast to int while returning in the tuple.
There was a problem hiding this comment.
Yeah...
Actually, typename std::iterator_traits<GroupIdIterator>::value_type should be int here (Group IDs or GPU IDs or process IDs are all assumed to be int). Using GroupIdIterator intead of int type pointer because I am using thrust::transform_iterator to compute group ID on the fly.
I'll add
static_assert(std::is_same_v(typename std::iterator_traits<GroupIdIterator>::value_type, int)); to make this clearer.
And I may modify operator()(size_t i) to operator()(int i) to avoid type casting... but I have mixed thoughts about this (I assume thrust::tabulate computes the ranges of i from thrust::distance() and i can potentially overflow int. Shouldn't happen unless we have more than 2^31-1 GPUs and that's the assumption for using 'int' here for GPU IDs. I guess explicitly casting this makes this intention more clearer than relying on implicit casting).
What do you think?
|
@gpucibot merge |
Pulls updates from PRs #2044, better be reviewed and merged after PR #2044 Mainly experimenting, not ready for review. Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) URL: #2062
Improve multi-node multi-GPU scalability of graph creation (especially the code computing renumber_map)
Fix an overflow bug when creating a graph with more than 2^31 vertices