Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable parallel mode #3875

Merged
merged 75 commits into from
Nov 21, 2023
Merged

Conversation

jnke2016
Copy link
Contributor

@jnke2016 jnke2016 commented Sep 25, 2023

This PR enables parallel mode

Closes https://github.com/rapidsai/graph_dl/issues/328

@jnke2016 jnke2016 requested review from a team as code owners September 25, 2023 10:42
@BradReesWork BradReesWork added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Sep 25, 2023
@BradReesWork BradReesWork added this to the 23.10 milestone Sep 25, 2023
@jnke2016 jnke2016 changed the title Enable batch mode Enable parallel mode Sep 25, 2023
Copy link
Collaborator

@ChuckHastings ChuckHastings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some C API comments to start with.

cpp/src/c_api/replicate_edgelist.cpp Outdated Show resolved Hide resolved
cpp/src/c_api/replicate_edgelist.cpp Outdated Show resolved Hide resolved
cpp/src/c_api/replicate_edgelist.cpp Outdated Show resolved Hide resolved
cpp/include/cugraph_c/graph_functions.h Outdated Show resolved Hide resolved
cpp/src/c_api/capi_helper.cu Outdated Show resolved Hide resolved
cpp/src/c_api/capi_helper.cu Outdated Show resolved Hide resolved
cpp/src/c_api/capi_helper.hpp Outdated Show resolved Hide resolved
cpp/src/c_api/replicate_edgelist.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@rlratzel rlratzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Joseph, I'm looking forward to having this capability. I found a few issues below.

python/cugraph/cugraph/structure/replication.py Outdated Show resolved Hide resolved
python/cugraph/cugraph/__init__.py Outdated Show resolved Hide resolved
python/cugraph/cugraph/structure/replication.py Outdated Show resolved Hide resolved
Comment on lines +50 to +71
fixture_params = gen_fixture_params_product(
(datasets, "graph_file"),
([True, False], "distributed"),
([True, False], "use_weights"),
([True, False], "use_edge_ids"),
([True, False], "use_edge_type_ids"),
)


@pytest.fixture(scope="module", params=fixture_params)
def input_combo(request):
"""
Simply return the current combination of params as a dictionary for use in
tests or other parameterized fixtures.
"""
return dict(
zip(
(
"graph_file",
"use_weights",
"use_edge_ids",
"use_edge_type_ids",
"distributed",
),
request.param,
)
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern isn't as useful here since there's only one test. It might be better for future maintainers if the test in this file just used regular pytest parameters.

python/pylibcugraph/pylibcugraph/replication.pyx Outdated Show resolved Hide resolved
python/pylibcugraph/pylibcugraph/replication.pyx Outdated Show resolved Hide resolved
python/pylibcugraph/pylibcugraph/replication.pyx Outdated Show resolved Hide resolved
python/pylibcugraph/pylibcugraph/replication.pyx Outdated Show resolved Hide resolved
@jnke2016 jnke2016 force-pushed the branch-23.10_batch-mode branch from 48e3f00 to 58014b9 Compare November 13, 2023 02:40
@jnke2016 jnke2016 force-pushed the branch-23.10_batch-mode branch from 0146339 to 3f09595 Compare November 20, 2023 15:18
Copy link
Contributor

@rlratzel rlratzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I found out why CI is failing for this PR.

@rlratzel
Copy link
Contributor

/merge

@rapids-bot rapids-bot bot merged commit 6b3d3e3 into rapidsai:branch-23.12 Nov 21, 2023
70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants