Remove CMake/C++ references to cugraph-ops #4744

ChuckHastings · 2024-11-01T15:41:54Z

Delete deprecated cugraph-ops functionality, clean up references to cugraph-ops.

…ugraph-ops

jameslamb

Is it safe to remove ALL references to cugraph-ops here? If so, there are many more:

git grep -i 'cugraph.*ops'

For example, all the uses of those now-removed CMake options in build scripts.

CI configs (click me)

cugraph/.github/workflows/build.yaml

Lines 79 to 81 in 050d524

    
           extra-repo: rapidsai/cugraph-ops 
        
           extra-repo-sha: branch-24.12 
        
           extra-repo-deploy-key: CUGRAPH_OPS_SSH_PRIVATE_DEPLOY_KEY

cugraph/.github/workflows/build.yaml

Lines 103 to 105 in 050d524

    
           extra-repo: rapidsai/cugraph-ops 
        
           extra-repo-sha: branch-24.12 
        
           extra-repo-deploy-key: CUGRAPH_OPS_SSH_PRIVATE_DEPLOY_KEY

cugraph/.github/workflows/pr.yaml

Lines 140 to 142 in 050d524

    
           extra-repo: rapidsai/cugraph-ops 
        
           extra-repo-sha: branch-24.12 
        
           extra-repo-deploy-key: CUGRAPH_OPS_SSH_PRIVATE_DEPLOY_KEY

cugraph/.github/workflows/pr.yaml

Lines 159 to 161 in 050d524

    
           extra-repo: rapidsai/cugraph-ops 
        
           extra-repo-sha: branch-24.12 
        
           extra-repo-deploy-key: CUGRAPH_OPS_SSH_PRIVATE_DEPLOY_KEY

cugraph/.github/workflows/test.yaml

Line 26 in 050d524

    
           symbol_exclusions: (cugraph::ops|hornet|void writeEdgeCountsKernel|void markUniqueOffsetsKernel)

build scripts (click me)

cugraph/build.sh

Line 46 in 050d524

--without_cugraphops

cugraph/build.sh

Line 110 in 050d524

BUILD_WITH_CUGRAPHOPS=ON

cugraph/build.sh

Line 271 in 050d524

-DUSE_CUGRAPH_OPS=${BUILD_WITH_CUGRAPHOPS} \

many more in build.sh

cugraph/ci/build_wheel_cugraph.sh

Line 31 in 050d524

    
           export SKBUILD_CMAKE_ARGS="-DDETECT_CONDA_ENV=OFF;-DFIND_CUGRAPH_CPP=OFF;-DCPM_cugraph-ops_SOURCE=${GITHUB_WORKSPACE}/cugraph-ops/${EXTRA_CMAKE_ARGS}"

cugraph/ci/build_wheel_pylibcugraph.sh

Line 18 in 050d524

    
           export SKBUILD_CMAKE_ARGS="-DDETECT_CONDA_ENV=OFF;-DFIND_CUGRAPH_CPP=OFF;-DCPM_cugraph-ops_SOURCE=${GITHUB_WORKSPACE}/cugraph-ops/${EXTRA_CMAKE_ARGS}"

dependency on 'libcugraphops' and 'pylibcugraphops' (click me)

cugraph/dependencies.yaml

Lines 23 to 24 in 050d524

    
             # Deprecate pylibcugraphops 
        
           - depends_on_pylibcugraphops

cugraph/dependencies.yaml

Lines 921 to 944 in 050d524

    
           depends_on_pylibcugraphops: 
        
             common: 
        
               - output_types: conda 
        
                 packages: 
        
                   - &pylibcugraphops_unsuffixed pylibcugraphops==24.12.*,>=0.0.0a0 
        
               - output_types: requirements 
        
                 packages: 
        
                   # pip recognizes the index as a global option for the requirements.txt file 
        
                   - --extra-index-url=https://pypi.nvidia.com 
        
                   - --extra-index-url=https://pypi.anaconda.org/rapidsai-wheels-nightly/simple 
        
             specific: 
        
               - output_types: [requirements, pyproject] 
        
                 matrices: 
        
                   - matrix: 
        
                       cuda: "12.*" 
        
                       cuda_suffixed: "true" 
        
                     packages: 
        
                       - pylibcugraphops-cu12==24.12.*,>=0.0.0a0 
        
                   - matrix: 
        
                       cuda: "11.*" 
        
                       cuda_suffixed: "true" 
        
                     packages: 
        
                       - pylibcugraphops-cu11==24.12.*,>=0.0.0a0 
        
                   - {matrix: null, packages: [*pylibcugraphops_unsuffixed]}

cugraph/dependencies.yaml

Lines 429 to 430 in 050d524

    
           # Deprecate libcugraphops 
        
           - libcugraphops==24.12.*,>=0.0.0a0

cugraph/conda/recipes/libcugraph/meta.yaml

Line 72 in 050d524

- libcugraphops ={{ minor_version }}

and several more, see that git grep output

cpp/CMakeLists.txt

bdice · 2024-11-04T15:33:21Z

@jameslamb Since CI has already passed here I am inclined to just accept this as-is and then iteratively rip out more. Maybe we can limit this PR's scope to CMake / C++ removal, and then do the removals you suggested (mostly CI, build scripts, dependencies, etc.) in a follow-up PR. I started to look into this as well and noticed there are quite a few references in the GNN code, which is being split out, so there may be potentially duplicate work that we should defer until after that split.

jameslamb · 2024-11-04T15:37:30Z

Ok yeah fair enough, let's stop here and do more in follow-ups.

bdice · 2024-11-04T15:42:29Z

python/cugraph/cugraph/tests/docs/test_doctests.py

@@ -186,11 +186,4 @@ def test_docstring(self, docstring):
                f"{docstring.name}:\n{doctest_stdout.getvalue()}"
            )
        except AssertionError:


Because this is no longer using specialized logic in the exception clause, the "except: raise" pattern is unneeded. We can remove the try entirely.

seunghwak

LGTM

ChuckHastings · 2024-11-08T21:24:29Z

/merge

This PR fixes a number of issues in the manifest: - cudf's dependency on rmm was accidentally removed in #221. Since rmm is header-only I guess nobody noticed the issue since it doesn't increase build times and only shows up if you attempt to make local changes to rmm and test them in cudf. - UCXX_ENABLE_PYTHON was removed in rapidsai/ucxx#257 - cuml depends on cuvs as of rapidsai/cuml#6085 - cugraph does not depend on cugraph-ops as of rapidsai/cugraph#4744 - Many packages were relying on transitively getting rmm or raft as dependencies, which causes problems when the intermediate dependency is removed. Extra `-D<package_name>_ROOT` variables in the CMake configure don't cause any problems so I made everything explicit. --------- Co-authored-by: Paul Taylor <[email protected]> Co-authored-by: Bradley Dice <[email protected]>

@acostadon

This repo should no longer need any build-time dependencies on cloning the `cugraph-ops` source code or pulling in its packages, as of these PRs: * #4744 * #4765 This PR proposes the following: * removing CI configuration related to cloning the `cugraph-ops` repo * removing CMake options related to `cugraph-ops` - `ALLOW_CLONE_CUGRAPH_OPS` - `CUGRAPH_EXCLUDE_CUGRAPH_OPS_FROM_ALL` - `CUGRAPH_USE_CUGRAPH_OPS_STATIC` - `USE_CUGRAPH_OPS` * removing dependencies on `pylibcugraphops` / `libcugraphops` in development scripts, conda recipes * removing the `cugraph-ops` docs, and references to them in other docs * removing unused source file `cpp/src/utilities/cugraph_ops_utils.hpp` * removing the `cugraph_ops` pytest marker ## Notes for Reviewers ### Benefits of these changes * faster CI * simplifies the pending work to introduce `libcugraph` wheel packages (rapidsai/rapids-wheels-planning#53) ### How I identified these changes ```shell git grep -i '_ops' git grep -i '\-ops' git grep -i 'cugraphops' ``` ### Why is this so big? I'd originally wanted to leave the docs-building stuff in place, but saw `docs-build` CI here failing because docs haven't been built from the `cugraph-ops` repo yet: https://github.com/rapidsai/cugraph/blob/c5d3d231c9cac361bf51246c13440a42eeda93b9/build.sh#L334-L341 Talked offline with @acostadon and @ChuckHastings , and we agreed that those docs could just be removed here. ### What is intentionally being left? * references in licenses (as some code from `cugraph-ops` is vendored here) * references in the docs for the GNN packages (`cugraph-dgl`, `cugraph-pyg`, `pylibwholegraph`), as those do still rely on `cugraph-ops` (their docs will eventually move out of this repo) Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Alex Barghi (https://github.com/alexbarghi-nv) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4784

Delete deprecated cugraph-ops functionality, clean up references to c…

9a411b8

…ugraph-ops

ChuckHastings self-assigned this Nov 1, 2024

github-actions bot added cuGraph CMake python labels Nov 1, 2024

ChuckHastings added improvement Improvement / enhancement to an existing function breaking Breaking change and removed cuGraph CMake python labels Nov 1, 2024

ChuckHastings marked this pull request as ready for review November 4, 2024 04:25

ChuckHastings requested review from a team as code owners November 4, 2024 04:25

jameslamb self-requested a review November 4, 2024 14:42

jameslamb reviewed Nov 4, 2024

View reviewed changes

cpp/CMakeLists.txt Show resolved Hide resolved

remove some cugraph-ops files that are no longer needed

83fea15

github-actions bot added cuGraph CMake python labels Nov 4, 2024

remove cugraph-ops options from build

eb758b8

ChuckHastings requested a review from a team as a code owner November 4, 2024 15:38

ChuckHastings requested a review from bdice November 4, 2024 15:38

jameslamb approved these changes Nov 4, 2024

View reviewed changes

bdice reviewed Nov 4, 2024

View reviewed changes

bdice approved these changes Nov 4, 2024

View reviewed changes

ChuckHastings changed the title ~~Remove references to cugraph-ops~~ Remove CMake/C++ references to cugraph-ops Nov 4, 2024

Trigger Build

8ef8c7e

seunghwak approved these changes Nov 5, 2024

View reviewed changes

rlratzel approved these changes Nov 6, 2024

View reviewed changes

rapids-bot bot merged commit 2d1189b into rapidsai:branch-24.12 Nov 8, 2024
140 checks passed

vyasr mentioned this pull request Nov 18, 2024

Various fixes to the manifest rapidsai/devcontainers#422

Merged

jameslamb mentioned this pull request Nov 25, 2024

remove all remaining uses of cugraph-ops #4784

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove CMake/C++ references to cugraph-ops #4744

Remove CMake/C++ references to cugraph-ops #4744

ChuckHastings commented Nov 1, 2024

jameslamb left a comment

bdice commented Nov 4, 2024 •

edited

Loading

jameslamb commented Nov 4, 2024

bdice Nov 4, 2024

seunghwak left a comment

ChuckHastings commented Nov 8, 2024

	extra-repo: rapidsai/cugraph-ops
	extra-repo-sha: branch-24.12
	extra-repo-deploy-key: CUGRAPH_OPS_SSH_PRIVATE_DEPLOY_KEY

	depends_on_pylibcugraphops:
	common:
	- output_types: conda
	packages:
	- &pylibcugraphops_unsuffixed pylibcugraphops==24.12.*,>=0.0.0a0
	- output_types: requirements
	packages:
	# pip recognizes the index as a global option for the requirements.txt file
	- --extra-index-url=https://pypi.nvidia.com
	- --extra-index-url=https://pypi.anaconda.org/rapidsai-wheels-nightly/simple
	specific:
	- output_types: [requirements, pyproject]
	matrices:
	- matrix:
	cuda: "12.*"
	cuda_suffixed: "true"
	packages:
	- pylibcugraphops-cu12==24.12.*,>=0.0.0a0
	- matrix:
	cuda: "11.*"
	cuda_suffixed: "true"
	packages:
	- pylibcugraphops-cu11==24.12.*,>=0.0.0a0
	- {matrix: null, packages: [*pylibcugraphops_unsuffixed]}

Remove CMake/C++ references to cugraph-ops #4744

Remove CMake/C++ references to cugraph-ops #4744

Conversation

ChuckHastings commented Nov 1, 2024

jameslamb left a comment

Choose a reason for hiding this comment

bdice commented Nov 4, 2024 • edited Loading

jameslamb commented Nov 4, 2024

bdice Nov 4, 2024

Choose a reason for hiding this comment

seunghwak left a comment

Choose a reason for hiding this comment

ChuckHastings commented Nov 8, 2024

bdice commented Nov 4, 2024 •

edited

Loading