Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bulk Loading Support for cuGraph-PyG #3170

Merged
merged 509 commits into from
Jan 31, 2023
Merged

Bulk Loading Support for cuGraph-PyG #3170

merged 509 commits into from
Jan 31, 2023

Conversation

alexbarghi-nv
Copy link
Member

@alexbarghi-nv alexbarghi-nv commented Jan 23, 2023 •
edited
Loading

Closes #3152
Depends on #3148 and #3159
This PR does support multiple trainers, even though there is no example for it.

ChuckHastings and others added 30 commits January 5, 2023 10:10
@ChuckHastings
address most PR comments
d47539b
@alexbarghi-nv
fixes
cd07a11
@alexbarghi-nv
merge
035498e
@alexbarghi-nv
do merge
6d0f7f0
@alexbarghi-nv
merge offsets
87bb91b
@alexbarghi-nv
remove test code
65dc4df
@alexbarghi-nv
type annotations
456ab7f
@ChuckHastings
finished addressing PR comments
8d6da9c
@alexbarghi-nv
remove unused import
f4cb88e
@alexbarghi-nv
update copyright
4420751
@alexbarghi-nv
update copyright year
8b5be47
@alexbarghi-nv
update copyright year
4ef2b2b
@alexbarghi-nv
update copyright year
d1079ea
@alexbarghi-nv
update copyright year
f715d14
@alexbarghi-nv
update copyright year
694b23e
@alexbarghi-nv
update copyright year
8179b1f
@alexbarghi-nv
update copyright year
bc6974e
@alexbarghi-nv
update copyright year
79abf44
@alexbarghi-nv
remove unused import
7af9b69
@alexbarghi-nv
do merge
db036ca
@alexbarghi-nv
update c++ code
2646563
@ChuckHastings
Merge branch 'branch-23.02' into update_uniform_sampling_api
eec3510
@alexbarghi-nv
fix typing
472f090
@alexbarghi-nv
resolve merge conflict
f04e00e
@alexbarghi-nv
fix array construction
52b2a4d
@alexbarghi-nv
Merge branch 'short-circuit-and-type-indices' of https://github.com/a…
cbd5c70 
…lexbarghi-nv/cugraph into short-circuit-and-type-indices
@alexbarghi-nv
fix type offsets
ebb2014
@alexbarghi-nv
fix index for non-cudf
a2120b6
@alexbarghi-nv
concatenate fix
6c29705
@alexbarghi-nv
fix cupy api issue
426b529
@rlratzel
Copy link
Contributor

/merge

@alexbarghi-nv alexbarghi-nv requested a review from a team as a code owner January 28, 2023 16:42
@rlratzel rlratzel mentioned this pull request Jan 30, 2023
Merged
8 tasks
@BradReesWork BradReesWork modified the milestones: 23.04, 23.02 Jan 30, 2023
ajschmidt8
ajschmidt8 reviewed Jan 30, 2023
View reviewed changes
Copy link
Member

@ajschmidt8 ajschmidt8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments.

If you're having GitHub Action CI-related issues, let me know and I can assist.

ci/cpu/build.sh Outdated Show resolved Hide resolved
ci/gpu/build.sh Outdated Show resolved Hide resolved
@alexbarghi-nv
Copy link
Member Author

@ajschmidt8 I reset the two dead files. CI is still pulling the wrong version of cugraph-pyg. Any idea on how to fix this?

@ajschmidt8
Copy link
Member

@ajschmidt8 I reset the two dead files. CI is still pulling the wrong version of cugraph-pyg. Any idea on how to fix this?

Let me check the logs and see what I can find.

@ajschmidt8
Copy link
Member

ajschmidt8 commented Jan 30, 2023 •
edited
Loading

@ajschmidt8 I reset the two dead files. CI is still pulling the wrong version of cugraph-pyg. Any idea on how to fix this?

@alexbarghi-nv, can you try adding --channel pyg to these lines?

These are the lines that install the workflow artifacts into the Python test job.

By default, our containers aren't configured to search the pyg channel for packages.

So since this PR adds pyg as a runtime dependency of the cugraph-pyg package, we'll need to explicitly add that channel when installing the cugraph-pyg package.

Right now conda is likely preferring the latest versions of the cugraph-pyg package that don't have this pyg dependency, which are the 23.04 packages at this time.

@alexbarghi-nv
Copy link
Member Author

@ajschmidt8 we have decided to drop the recipe change from this PR. Thanks for your help, hopefully we can get that done in another PR soon.

rapids-bot bot pushed a commit that referenced this pull request Jan 31, 2023
@VibhuJawa
DGL Dataloader (#3181)
a51bfa7 
This PR depends upon ~#3148 (#3170)

This PR closes #3154


- [x] Works on SG Homogenous Graph (OBGN-Products) 
- [x] Works on SG Heterogenous Graphs  (OBGN-Mag) 
- [x] Works on MG Homogenous Graphs  (OBGN-Products, OBGN-PAPERS100M (1 Billion edge graph)) 
- [x] Works on MG Heterogenous Graphs (OBGN-Mag) 

Tests to add:
- [x] Tests on SG Homogenous Graphs
- [x] Tests  on SG Heterogenous Graphs
- [x] Tests  on MG Homogenous Graphs
- [x] Tests on MG Heterogenous Graphs

Maybe Todo: 
<s>- [ ] Test Multi-trainer examples and verification </s> (Will do a follow up)

Authors:
  - Vibhu Jawa (https://github.com/VibhuJawa)
  - Rick Ratzel (https://github.com/rlratzel)
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #3181
@rapids-bot rapids-bot bot merged commit 867a0ac into rapidsai:branch-23.02 Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ajschmidt8 ajschmidt8 ajschmidt8 left review comments

@VibhuJawa VibhuJawa VibhuJawa approved these changes

@rlratzel rlratzel rlratzel approved these changes

Assignees

@alexbarghi-nv alexbarghi-nv

Labels
feature request New feature or request non-breaking Non-breaking change
Projects
None yet
Milestone
23.02
Development

Successfully merging this pull request may close these issues.

[IMP] Implement a Sampler and Loader for BulkSampler in cugraph-PyG
7 participants
@alexbarghi-nv @codecov-commenter @VibhuJawa @rlratzel @ajschmidt8 @BradReesWork @ChuckHastings