[2/n] Add setting function to set seeds to the graph #894
Conversation
facebook-github-bot
added
the
CLA Signed
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| return datapipe | ||
|
|
||
|
|
||
| def _set_graph_seeds(datapipe: DataPipe, seed_generator: torch.Generator, worker_id: int = 0) -> DataPipe: |
There was a problem hiding this comment.
I am using a private name because I don't like the name of this function. LMK if you have any better idea.
There was a problem hiding this comment.
_set_worker_seed_set_worker_seed_for_dp_graph
| return datapipe | ||
|
|
||
|
|
||
| def _set_graph_seeds(datapipe: DataPipe, seed_generator: torch.Generator, worker_id: int = 0) -> DataPipe: |
There was a problem hiding this comment.
_set_worker_seed_set_worker_seed_for_dp_graph
ejguan
force-pushed
the
shuffle_after_sharding
branch
from
db08b88 to
fd7808a
Compare
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Changes
list_dpswith a new argument (to make sure allexclusive)DataPipeswill be excluded even though they are the predecessors of the non-excludedDataPipe.list_dpsto use BFS to return the list ofDataPipes._set_worker_seed_for_dp_graphto set seeds forDataPipesbeforesharding_filterwith the same seed generator and set different seeds forDataPipesaftersharding_filterwith the worker-local seed generator.Step 2 for #885