Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Add cugraph+UCX build instructions #3088

Merged
merged 11 commits into from
Jan 4, 2023

Conversation

VibhuJawa
Copy link
Member

@VibhuJawa VibhuJawa commented Dec 19, 2022

This PR adds the instructions to build cugraph+UCX for benchmarking .

The current conda build does not support IB as well has challenges using NVLINK . This PR adds the dockefile as well as build instructions and test to streamline benchmarking effort.

The expected speedup on sampling is around 8x by using the UCX souce build and a 20x better bandwidth with compute.

See README for instructions .

Benchmarks

dask.compute bandwidth

With the standard conda build :

python3 test_client_bandwidth.py 

Getting 1,000,000 rows  of size 0.01 took = 168.32900047302246 ms
Bandwidth = 0.0594 gb/s
Getting 2,000,000 rows  of size 0.02 took = 260.2977991104126 ms
Bandwidth = 0.0768 gb/s
Getting 4,000,000 rows  of size 0.04 took = 288.5307312011719 ms
Bandwidth = 0.1386 gb/s
Getting 25,000,000 rows  of size 0.28 took = 1051.6365766525269 ms
Bandwidth = 0.2663 gb/s
----------------------------------------Completed Test----------------------------------------

With this PRs Build :

python3 test_client_bandwidth.py

Getting 1,000,000 rows  of size 0.01 took = 14.061594009399414 ms
Bandwidth = 0.7112 gb/s
Getting 2,000,000 rows  of size 0.02 took = 19.601154327392578 ms
Bandwidth = 1.0203 gb/s
Getting 4,000,000 rows  of size 0.04 took = 15.92099666595459 ms
Bandwidth = 2.5124 gb/s
Getting 25,000,000 rows  of size 0.28 took = 52.902913093566895 ms
Bandwidth = 5.2927 gb/s
----------------------------------------Completed Test----------------------------------------

cugraph sampling benchmark

With the standard conda build :

python3 test_cugraph_sampling.py

Sampling 1,000 took = 78.96044254302979 ms
Sampling 10,000 took = 189.63768482208252 ms
Sampling 100,000 took = 1084.9100351333618 ms
----------------------------------------Completed Test----------------------------------------

With this PRs Build :

python3 test_cugraph_sampling.py


Sampling 1,000 took = 73.62375259399414 ms
Sampling 10,000 took = 87.1060848236084 ms
Sampling 100,000 took = 134.7532033920288 ms
----------------------------------------Completed Test----------------------------------------

@VibhuJawa VibhuJawa self-assigned this Dec 19, 2022
@VibhuJawa VibhuJawa added non-breaking Non-breaking change feature request New feature or request labels Dec 19, 2022
@BradReesWork BradReesWork added this to the 23.02 milestone Dec 19, 2022
@VibhuJawa VibhuJawa requested a review from a team as a code owner December 19, 2022 22:55
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Member

@alexbarghi-nv alexbarghi-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs copyright fix

benchmarks/shared/build_cugraph_ucx/build-ucx.sh Outdated Show resolved Hide resolved
benchmarks/shared/build_cugraph_ucx/cugraph_ucx.dockerfile Outdated Show resolved Hide resolved
Copy link
Contributor

@rlratzel rlratzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I do have a question and a suggestion but I'll also approve as-is.

@codecov-commenter
Copy link

codecov-commenter commented Jan 3, 2023

Codecov Report

Base: 58.86% // Head: 57.14% // Decreases project coverage by -1.71% ⚠️

Coverage data is based on head (88a8865) compared to base (2ae4b61).
Patch coverage: 43.95% of modified lines in pull request are covered.

Additional details and impacted files
@@               Coverage Diff                @@
##           branch-23.02    #3088      +/-   ##
================================================
- Coverage         58.86%   57.14%   -1.72%     
================================================
  Files               131      148      +17     
  Lines              7848     9302    +1454     
================================================
+ Hits               4620     5316     +696     
- Misses             3228     3986     +758     
Impacted Files Coverage Δ
...-service/server/cugraph_service_server/__main__.py 0.00% <0.00%> (ø)
...ure/graph_implementation/simpleDistributedGraph.py 14.45% <0.00%> (-0.45%) ⬇️
...ugraph/cugraph/dask/structure/mg_property_graph.py 11.97% <9.09%> (-0.12%) ⬇️
...-service/server/cugraph_service_server/__init__.py 25.00% <14.28%> (ø)
...rvice_server/testing/benchmark_server_extension.py 40.38% <40.38%> (ø)
...ice/server/cugraph_service_server/testing/utils.py 74.13% <41.17%> (ø)
...vice/client/cugraph_service_client/remote_graph.py 43.75% <55.55%> (ø)
...e/server/cugraph_service_server/cugraph_handler.py 43.81% <70.31%> (ø)
python/pylibcugraph/pylibcugraph/testing/utils.py 17.77% <0.00%> (-12.66%) ⬇️
python/cugraph/cugraph/testing/utils.py 71.42% <0.00%> (-4.19%) ⬇️
... and 21 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@VibhuJawa
Copy link
Member Author

@rlratzel , Addressed your reviews, please merge if this is good to merge.

@rlratzel
Copy link
Contributor

rlratzel commented Jan 3, 2023

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 976da08 into rapidsai:branch-23.02 Jan 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants