Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reapply "defaults: make dmabuf opt-in" #739

Merged
merged 1 commit into from
Dec 5, 2024
Merged

Reapply "defaults: make dmabuf opt-in" #739

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions include/nccl_ofi_param.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,14 +272,15 @@ OFI_NCCL_PARAM_INT(disable_gdr_required_check, "DISABLE_GDR_REQUIRED_CHECK", 0);
* Unfortunately, the plugin needs to signal DMABUF support or lack thereof back
* to NCCL prior to having an opportuntiy to make any any memory registrations.
* This ultimately means that the plugin will opimistically assume DMA-BUF is
* viable on all FI_HMEM providers beyond libfabric 1.20.
* viable on all FI_HMEM providers beyond libfabric 1.20, if not for this param.
*
* If dmabuf registrations fail, (ie: if ibv_reg_dmabuf_mr cannot be resolved),
* the plugin has no freedom to renegotiate DMABUF support with NCCL, and so it
* is fatal. Under those conditions, users should set this environment variable
* to force NCCL to avoid providing dmabuf file desciptors.
* is fatal. Under those conditions, users should ensure that they have set this
* environment variable to '1' to force NCCL to avoid providing dmabuf file
* desciptors. This is the default, pending perf investigations.
*/
OFI_NCCL_PARAM_INT(disable_dmabuf, "DISABLE_DMABUF", 0);
OFI_NCCL_PARAM_INT(disable_dmabuf, "DISABLE_DMABUF", 1);

/*
* Messages sized larger than this threshold will be striped across multiple rails
Expand Down
Loading