[QST]: How to use Multi-GPU computation in cugraph

[whuLames]

What is your question?

I run the code in there for using Multi-GPU computation,but the code have the following error:
/root/miniconda3/envs/cugraph/lib/python3.9/site-packages/dask_cudf/io/csv.py:79: FutureWarning: chunksize is deprecated and will be removed in the future. Please use blocksize instead.
warn(
0.04763150215148926
Exception ignored in: <function Future.del at 0x7f1d1c7e9e50>
Traceback (most recent call last):
File "/root/miniconda3/envs/cugraph/lib/python3.9/site-packages/distributed/client.py", line 511, in del
TypeError: 'NoneType' object is not callable
Exception ignored in: <function Future.del at 0x7f1d1c7e9e50>
Traceback (most recent call last):
File "/root/miniconda3/envs/cugraph/lib/python3.9/site-packages/distributed/client.py", line 511, in del
TypeError: 'NoneType' object is not callable

I have a couple questions about the code above：

1. This error appears to be an error that occurs when the client closes, what exactly is the reason for this error to occur?
2. If this error is an error that occurs when the client is closed, does this error not affect the results of the graph algorithm calculations under multi-gpu?
3. How to count the runtime of graph algorithms on multiple GPUs？
   t1 = time.time()
pr_df = dask_cugraph.pagerank(G)
t2 = time.time()
   Is it possible to count the execution time under multi-GPU in this way？
4. How to specify a specific number of GPUs for computation when the machine has eight GPUs？

Thanks !

Code of Conduct

• I agree to follow cuGraph's Code of Conduct
• I have searched the open issues and have found no duplicates for this question

[acostadon]

We are looking into this. Will get an answer here as soon as possible.

[alexbarghi-nv]

You need to call .compute() to get results.
i.e.

t1 = time.time()
pr_df = dask_cugraph.pagerank(G).compute()
t2 = time.time()

That error is from not properly closing the dask client. It occurs at shutdown, and isn't affecting any results you're getting. You should be able to remove it by calling client.close()

[acostadon]

Thanks for that answer, Alex

Zhang, the example you cited has been updated to avoid future confusion. The changes will be merged in the 24.04 release.

Unless you need additional support on this matter, we will close your question within the week.

[whuLames]

Thanks for that answer, Alex

Zhang, the example you cited has been updated to avoid future confusion. The changes will be merged in the 24.04 release.

Unless you need additional support on this matter, we will close your question within the week.

Sorry for late feedback
I've found that cugraph's multi-GPU doesn't scale the size of graph data that can be processed, graph data that appears oom under a single card still appears oom under multiple cards

I want to know why

[alexbarghi-nv]

@whuLames how large is the graph you're trying to read? It could just be larger than what two cards can support. Or you may be running into OOM in the preprocessing stage. You could try enabling cudf spilling, or using managed memory with rmm.reinitialize(...).

[alexbarghi-nv]

@whuLames I also just noticed your question #4 - to limit the GPUs used, you can set the CUDA_VISIBLE_DEVICES environment variable. For instance, if you wanted to use GPUs 0, 2, 4, and 6, then you would set CUDA_VISIBLE_DEVICES=0,2,4,6