Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

cub device scan is not deterministic as described in the documentation #454

Closed
zasdfgbnm opened this issue Apr 5, 2022 · 3 comments
Closed
Labels
helps: pytorch Helps or needed by PyTorch. P1: should have Necessary, but not critical. type: bug: functional Does not work as intended.

Comments

@zasdfgbnm
Copy link
Contributor

Issue originally reported at pytorch/pytorch#75240
To repro, see: #453

@wence-
Copy link

wence- commented Apr 6, 2022

FWIW, 7a8320c updates the docs to indicate that these operations are not deterministic run-to-run. But it looks like there hasn't been a release to build them since then?

@alliepiper
Copy link
Collaborator

Unfortunately this issue has been popping up a lot recently in several algorithms, scan and reduce variants in particular. It appears that this documentation has been incorrect for quite some time, as these algorithms are implemented using techniques like decoupled lookback that are completely non-deterministic.

I'm aware of the situation and would like to fix it, but currently lack the resources needed to address these problems. This situation should hopefully improve in the next few months and ideally we'll be able to provide deterministic versions of these algorithms at some point.

@alliepiper alliepiper added type: bug: functional Does not work as intended. P1: should have Necessary, but not critical. helps: pytorch Helps or needed by PyTorch. labels Apr 6, 2022
@alliepiper
Copy link
Collaborator

Wrote up a summary of the issue and some proposed solutions in NVIDIA/cccl#886. Feedback is welcome -- closing this issue to consolidate discussion to the new issue.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
helps: pytorch Helps or needed by PyTorch. P1: should have Necessary, but not critical. type: bug: functional Does not work as intended.
Projects
None yet
Development

No branches or pull requests

3 participants