Update build system and docs to new minimum cudf-0.20 requirements #7780

robertmaynard · 2021-03-31T13:15:13Z

cudf-0.20 increase the minimum requirements in the following way:

GCC version 9.0+ is required
CUDA and C++ code now is compiled with -std=c++17
We require CUDA Toolkit version 11.0 or greater

This updates the build-system and the README with these new requirements

jrhemstad · 2021-03-31T13:48:53Z

Can we include updating to C++17 with this change?

robertmaynard · 2021-03-31T14:07:29Z

Can we include updating to C++17 with this change?

Will do.

CONTRIBUTING.md

robertmaynard · 2021-03-31T16:07:21Z

It is expected that all the builds won't pass since we still have CUDA 10.1 and 10.2 testers

karthikeyann · 2021-03-31T16:53:30Z

It is expected that all the builds won't pass since we still have CUDA 10.1 and 10.2 testers

Will this PR include update to those CI testers as well?

robertmaynard · 2021-03-31T19:09:38Z

It is expected that all the builds won't pass since we still have CUDA 10.1 and 10.2 testers

Will this PR include update to those CI testers as well?

Currently working on how to update CI

harrism · 2021-03-31T23:24:57Z

@robertmaynard can you confirm this will also resolve #7695 ?

harrism · 2021-03-31T23:56:28Z

CUDA 11.0 supports Ubuntu 18.0, where the default compiler is GCC 8.x. GCC 8.x supports C++17. So shouldn't our min GCC be 8.x, not 9.x?

https://docs.nvidia.com/cuda/archive/11.0/cuda-installation-guide-linux/index.html

kkraus14 · 2021-03-31T23:59:18Z

CUDA 11.0 supports Ubuntu 18.0, where the default compiler is GCC 8.x. GCC 8.x supports C++17. So shouldn't our min GCC be 8.x, not 9.x?

https://docs.nvidia.com/cuda/archive/11.0/cuda-installation-guide-linux/index.html

I believe not all C++17 features are supported until GCC 9.x. Additionally, conda uses gcc 9.3.0 for everything which means it's not safe for us to mix gcc 8 and conda in CI.

jlowe · 2021-04-01T00:23:15Z

I believe not all C++17 features are supported until GCC 9.x. Additionally, conda uses gcc 9.3.0 for everything which means it's not safe for us to mix gcc 8 and conda in CI.

According to https://gcc.gnu.org/projects/cxx-status.html everything in the C++17 standard is covered by GCC 8.

Conda may require the packages to be built with a GCC 9.3.0 toolchain, but it would be very nice to not force 9.3.0 on everyone else as a result if all we really care about source-wise is C++17 compliance.

kkraus14 · 2021-04-01T00:32:47Z

I may be misremembering a discussion we had surrounding this where it may have been gcc 7 vs 8 instead of 8 vs 9. If we set our minimum version to gcc 8 that works for me.

jrhemstad · 2021-04-01T00:48:29Z

<memory_resource> wasn't added to libstdc++ until GCC 9. It would be kinda nice to be able to rely on that in the future.

harrism · 2021-04-01T01:37:18Z

<memory_resource> wasn't added to libstdc++ until GCC 9. It would be kinda nice to be able to rely on that in the future.

Damn. That would be nice. BTW, for reference https://en.cppreference.com/w/cpp/compiler_support

harrism · 2021-04-01T01:38:17Z

The question is whether our non-conda users are going to complain about upgrading their compiler.

kkraus14 · 2021-04-01T02:15:45Z

The question is whether our non-conda users are going to complain about upgrading their compiler.

I believe we've required gcc 7.x for a while now which required installing an updated compiler on CentOS and Ubuntu 16.04. We got a few one off issues here and there related to compiler versions, but nothing significant.

I suspect we're going to get much more backlash from dropping CUDA 10.x.

robertmaynard · 2021-04-01T13:55:42Z

CUDA 11.0 supports Ubuntu 18.0, where the default compiler is GCC 8.x. GCC 8.x supports C++17. So shouldn't our min GCC be 8.x, not 9.x?
https://docs.nvidia.com/cuda/archive/11.0/cuda-installation-guide-linux/index.html

I believe not all C++17 features are supported until GCC 9.x. Additionally, conda uses gcc 9.3.0 for everything which means it's not safe for us to mix gcc 8 and conda in CI.

In GCC 8, the <filesystem> implementation was in a separate library ( -lstdc++fs ), while in 9 it is part of the libstdc++. GCC 9 is the first release where libstdc++ C++17 support was no longer experimental ( https://gcc.gnu.org/gcc-9/changes.html ).

robertmaynard · 2021-04-01T14:01:57Z

@robertmaynard can you confirm this will also resolve #7695 ?

It does fix the documentation being out of date, but not in the ideal way ( a single central RAPIDS requirements doc ).

jrhemstad · 2021-04-01T14:42:51Z

The fact that both <filesystem> and <memory_resource> aren't available until gcc9 seals the deal imo.

robertmaynard · 2021-04-01T19:12:45Z

rerun tests

robertmaynard · 2021-04-02T13:42:54Z

rerun tests

kkraus14

CMake / docs lgtm

mythrocks · 2021-04-06T19:56:37Z

Given that the requirements of CUDA version and min compiler version have both changed, should we mark this as a breaking change?

codereport

Upgraded NVIDIA drivers/CUDA and pulled down changes and got this PR / branch-0.20 building 👍

ttnghia · 2021-04-07T12:52:32Z

Rerun tests.

robertmaynard · 2021-04-07T16:32:39Z

Looks like a single consistent failure of:

10:33:04  1 FAILED TEST
10:33:04 + remove_libcudf_kernel_cache_dir
10:33:04 + EXITCODE=1
10:33:04 + gpuci_logger 'TRAP: Removing kernel cache dir: /workspace/.jitify-cache'
10:33:04 
10:33:04 gpuCI logger » [04/07/2021 14:33:02]
10:33:04 ┌─────────────────────────────────────────────────────────────────┐
10:33:04 |    TRAP: Removing kernel cache dir: /workspace/.jitify-cache    |
10:33:04 └─────────────────────────────────────────────────────────────────┘

Which seems to be independent of these changes.

kkraus14 · 2021-04-07T16:46:25Z

I think the failure is actually: https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci-v0.20/job/cudf/job/prb/job/cudf-gpu-test/CUDA=11.0,GPU_LABEL=gpu-a100,OS=ubuntu16.04,PYTHON=3.7/33/testReport/(root)/ListsColumnTest/ConcatenateEmptyLists/

robertmaynard · 2021-04-07T18:55:03Z

I think the failure is actually: https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci-v0.20/job/cudf/job/prb/job/cudf-gpu-test/CUDA=11.0,GPU_LABEL=gpu-a100,OS=ubuntu16.04,PYTHON=3.7/33/testReport/(root)/ListsColumnTest/ConcatenateEmptyLists/

Looks #7892 addresses that issue. So once that is merged in I will rebase this

Starting with rmm-0.20 the minimum allowed C++ and CUDA language level is C++17, and GCC-9.

codecov · 2021-04-08T00:12:04Z

Codecov Report

Merging #7780 (1a2865d) into branch-0.20 (599f62d) will increase coverage by 0.44%.
The diff coverage is 87.68%.

❗ Current head 1a2865d differs from pull request most recent head faea028. Consider uploading reports for the commit faea028 to get more accurate results

@@               Coverage Diff               @@
##           branch-0.20    #7780      +/-   ##
===============================================
+ Coverage        82.30%   82.74%   +0.44%     
===============================================
  Files              101      103       +2     
  Lines            17053    17721     +668     
===============================================
+ Hits             14035    14663     +628     
- Misses            3018     3058      +40

Impacted Files	Coverage Δ
python/cudf/cudf/utils/dtypes.py	`83.44% <46.66%> (-6.45%)`	⬇️
python/cudf/cudf/core/groupby/groupby.py	`92.41% <78.57%> (-1.04%)`	⬇️
python/cudf/cudf/core/column/lists.py	`87.41% <80.00%> (+0.19%)`	⬆️
python/dask_cudf/dask_cudf/backends.py	`89.58% <85.71%> (-0.05%)`	⬇️
python/cudf/cudf/core/column/struct.py	`96.29% <86.66%> (-3.71%)`	⬇️
python/cudf/cudf/core/index.py	`93.04% <88.09%> (+0.01%)`	⬆️
python/cudf/cudf/core/column/decimal.py	`92.92% <91.48%> (-0.92%)`	⬇️
python/cudf/cudf/core/column/interval.py	`91.11% <92.30%> (+0.48%)`	⬆️
python/cudf/cudf/core/column/column.py	`87.99% <92.59%> (+0.56%)`	⬆️
python/cudf/cudf/core/join/join.py	`96.75% <93.75%> (+0.24%)`	⬆️
... and 61 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a995e5...faea028. Read the comment docs.

jrhemstad · 2021-04-08T13:30:52Z

@gpucibot merge

…#1552) Close #1528 Following cuDF (rapidsai/cudf#7780) cugraph-0.20 increase the minimum requirements in the following way: GCC version 9.3+ is required CUDA and C++ code now is compiled with -std=c++17 We require CUDA Toolkit version 11.0 or greater This updates the build-system and the README with these new requirements Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - AJ Schmidt (https://github.com/ajschmidt8) - Brad Rees (https://github.com/BradReesWork) URL: #1552

robertmaynard requested a review from a team as a code owner March 31, 2021 13:15

github-actions bot added CMake CMake build issue libcudf Affects libcudf (C++/CUDA) code. labels Mar 31, 2021

robertmaynard changed the title ~~Update build system and docs to state cuda 11 required~~ Update build system and docs to new minimum rmm-0.20 requirements Mar 31, 2021

robertmaynard changed the title ~~Update build system and docs to new minimum rmm-0.20 requirements~~ Update build system and docs to new minimum cudf-0.20 requirements Mar 31, 2021

robertmaynard force-pushed the update_build_system_and_docs_to_state_cuda_11_required branch from 2a8a42b to 468ecfc Compare March 31, 2021 14:50

jrhemstad reviewed Mar 31, 2021

View reviewed changes

CONTRIBUTING.md Show resolved Hide resolved

jrhemstad mentioned this pull request Mar 31, 2021

Add groupby product support #7763

Merged

jlowe mentioned this pull request Mar 31, 2021

update Java bindings version to 0.20 [skip ci] #7747

Merged

robertmaynard force-pushed the update_build_system_and_docs_to_state_cuda_11_required branch from fbc0b5c to 71be28e Compare April 2, 2021 14:31

robertmaynard requested a review from kkraus14 April 6, 2021 17:55

kkraus14 approved these changes Apr 6, 2021

View reviewed changes

jrhemstad approved these changes Apr 6, 2021

View reviewed changes

kkraus14 added breaking Breaking change feature request New feature or request labels Apr 6, 2021

codereport approved these changes Apr 6, 2021

View reviewed changes

robertmaynard force-pushed the update_build_system_and_docs_to_state_cuda_11_required branch from 33b3552 to d6132a8 Compare April 7, 2021 12:54

robertmaynard added 6 commits April 7, 2021 17:39

cudf README updated to express 0.20 requirements of CUDA 11.X

c0f8d2f

Remove CMake logic for unsupported CUDA Toolkit versions

ca14d9d

Update cudf minimum C++ and CUDA language level to 17.

6ef76c6

Starting with rmm-0.20 the minimum allowed C++ and CUDA language level is C++17, and GCC-9.

allow lash_hash_file.cu to compiler under C++17

aa68c38

update NVIDIA Driver version to minimums for enhanced compatibility

4595559

jitify_preprocess now compiles jit files with C++17

faea028

robertmaynard force-pushed the update_build_system_and_docs_to_state_cuda_11_required branch from d6132a8 to faea028 Compare April 7, 2021 21:39

ttnghia mentioned this pull request Apr 7, 2021

Struct binary search (lower_bound/upper_bound) #7865

Merged

rapids-bot bot merged commit acf155d into rapidsai:branch-0.20 Apr 8, 2021

robertmaynard deleted the update_build_system_and_docs_to_state_cuda_11_required branch April 8, 2021 13:36

jrhemstad mentioned this pull request Apr 8, 2021

[FEA] Remove CUDA 10.x compiler bug work arounds #7913

Closed

seunghwak mentioned this pull request Apr 22, 2021

Update build system and docs to new minimum cugraph-0.20 requirements rapidsai/cugraph#1552

Merged

vyasr mentioned this pull request Oct 28, 2024

Upgrade RAPIDS to building with C++20 rapidsai/build-planning#113

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update build system and docs to new minimum cudf-0.20 requirements #7780

Update build system and docs to new minimum cudf-0.20 requirements #7780

robertmaynard commented Mar 31, 2021 •

edited

Loading

jrhemstad commented Mar 31, 2021

robertmaynard commented Mar 31, 2021

robertmaynard commented Mar 31, 2021

karthikeyann commented Mar 31, 2021

robertmaynard commented Mar 31, 2021

harrism commented Mar 31, 2021

harrism commented Mar 31, 2021 •

edited

Loading

kkraus14 commented Mar 31, 2021

jlowe commented Apr 1, 2021

kkraus14 commented Apr 1, 2021

jrhemstad commented Apr 1, 2021 •

edited

Loading

harrism commented Apr 1, 2021

harrism commented Apr 1, 2021

kkraus14 commented Apr 1, 2021

robertmaynard commented Apr 1, 2021 •

edited

Loading

robertmaynard commented Apr 1, 2021

jrhemstad commented Apr 1, 2021

robertmaynard commented Apr 1, 2021

robertmaynard commented Apr 2, 2021

kkraus14 left a comment

mythrocks commented Apr 6, 2021

codereport left a comment

ttnghia commented Apr 7, 2021

robertmaynard commented Apr 7, 2021

kkraus14 commented Apr 7, 2021

robertmaynard commented Apr 7, 2021

codecov bot commented Apr 8, 2021 •

edited

Loading

jrhemstad commented Apr 8, 2021

Update build system and docs to new minimum cudf-0.20 requirements #7780

Update build system and docs to new minimum cudf-0.20 requirements #7780

Conversation

robertmaynard commented Mar 31, 2021 • edited Loading

jrhemstad commented Mar 31, 2021

robertmaynard commented Mar 31, 2021

robertmaynard commented Mar 31, 2021

karthikeyann commented Mar 31, 2021

robertmaynard commented Mar 31, 2021

harrism commented Mar 31, 2021

harrism commented Mar 31, 2021 • edited Loading

kkraus14 commented Mar 31, 2021

jlowe commented Apr 1, 2021

kkraus14 commented Apr 1, 2021

jrhemstad commented Apr 1, 2021 • edited Loading

harrism commented Apr 1, 2021

harrism commented Apr 1, 2021

kkraus14 commented Apr 1, 2021

robertmaynard commented Apr 1, 2021 • edited Loading

robertmaynard commented Apr 1, 2021

jrhemstad commented Apr 1, 2021

robertmaynard commented Apr 1, 2021

robertmaynard commented Apr 2, 2021

kkraus14 left a comment

Choose a reason for hiding this comment

mythrocks commented Apr 6, 2021

codereport left a comment

Choose a reason for hiding this comment

ttnghia commented Apr 7, 2021

robertmaynard commented Apr 7, 2021

kkraus14 commented Apr 7, 2021

robertmaynard commented Apr 7, 2021

codecov bot commented Apr 8, 2021 • edited Loading

Codecov Report

jrhemstad commented Apr 8, 2021

robertmaynard commented Mar 31, 2021 •

edited

Loading

harrism commented Mar 31, 2021 •

edited

Loading

jrhemstad commented Apr 1, 2021 •

edited

Loading

robertmaynard commented Apr 1, 2021 •

edited

Loading

codecov bot commented Apr 8, 2021 •

edited

Loading