Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transform Reduce E test #1798

Merged
merged 5 commits into from
Sep 15, 2021
Merged

Conversation

kaatish
Copy link
Collaborator

@kaatish kaatish commented Aug 31, 2021

This PR adds multi-gpu testing for transform_reduce_e primitive.

A bug was fixed in transform_reduce_e where shared memory was being allocated inside a device function for the purpose of using it with cub::BlockReduce.

Utilities in dataframe_buffer.cuh were cleaned up. dataframe_buffer related functions with the exception of allocation functions no longer require an explicit template parameter.

cpp/tests/community/mg_louvain_helper.cu Outdated Show resolved Hide resolved
cpp/tests/community/mg_louvain_test.cpp Outdated Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Outdated Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Outdated Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Outdated Show resolved Hide resolved
@kaatish kaatish added 2 - In Progress non-breaking Non-breaking change improvement Improvement / enhancement to an existing function and removed improvement Improvement / enhancement to an existing function labels Sep 1, 2021
@kaatish kaatish self-assigned this Sep 1, 2021
@kaatish kaatish added this to the 21.10 milestone Sep 1, 2021
@kaatish kaatish added bug Something isn't working improvement Improvement / enhancement to an existing function labels Sep 3, 2021
@kaatish kaatish requested a review from seunghwak September 3, 2021 20:12
@kaatish kaatish added 3 - Ready for Review and removed 2 - In Progress bug Something isn't working labels Sep 3, 2021
@codecov-commenter
Copy link

codecov-commenter commented Sep 3, 2021

Codecov Report

Merging #1798 (63ce53f) into branch-21.10 (bf64c2c) will increase coverage by 9.73%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.10    #1798      +/-   ##
================================================
+ Coverage         59.85%   69.58%   +9.73%     
================================================
  Files                77      137      +60     
  Lines              3547     8581    +5034     
================================================
+ Hits               2123     5971    +3848     
- Misses             1424     2610    +1186     
Impacted Files Coverage Δ
python/cugraph/dask/community/__init__.py
python/cugraph/traversal/bfs.py
python/cugraph/internals/__init__.py
python/cugraph/dask/link_analysis/pagerank.py
python/cugraph/__init__.py
python/cugraph/utilities/__init__.py
python/cugraph/layout/__init__.py
python/cugraph/link_prediction/woverlap.py
python/cugraph/centrality/katz_centrality.py
python/cugraph/proto/components/scc.py
... and 204 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df2d81d...63ce53f. Read the comment docs.

Copy link
Contributor

@seunghwak seunghwak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the updates you made to dataframe_buffer.cuh!!!

It might be better to update the test code to be readable by users without strong expertise in template metaprogramming (don't go too far if it requires significantly sacrificing code quality... and I understand it might not be possible... but just in case if possible). I am just a bit worried about scaring potential graph primitives users who are not very familiar with TMP.

This PR looks really good otherwise.

e_op_result_sum =
block_reduce_edge_op_result<e_op_result_t, transform_reduce_e_for_all_block_size>().compute(
e_op_result_sum);
e_op_result_sum = BlockReduce(temp_storage).Reduce(e_op_result_sum, edge_property_add);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed, if block_reduce_edge_op_result can serve what we need, we'd better delete this and directly use cub everywhere for consistency.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

block_reduce_edge_op_result is still being used by copy_v_transform_reduce_nbr and therefore cannot be removed yet. It should be removed though because it is buggy. Perhaps in another PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... you can remove it when you write test cases for copy_v_transform_reduce_nbr.

cpp/tests/prims/mg_transform_reduce_e.cu Outdated Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Outdated Show resolved Hide resolved
cpp/tests/prims/mg_transform_reduce_e.cu Outdated Show resolved Hide resolved
@kaatish kaatish requested a review from seunghwak September 14, 2021 23:06
Copy link
Contributor

@seunghwak seunghwak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great and thanks for the updates!!!

e_op_result_sum =
block_reduce_edge_op_result<e_op_result_t, transform_reduce_e_for_all_block_size>().compute(
e_op_result_sum);
e_op_result_sum = BlockReduce(temp_storage).Reduce(e_op_result_sum, edge_property_add);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... you can remove it when you write test cases for copy_v_transform_reduce_nbr.

@BradReesWork
Copy link
Member

@gpucibot merge

@rapids-bot rapids-bot bot merged commit fd484d5 into rapidsai:branch-21.10 Sep 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants