nvidia dali support

[smallzzy]

🚀 Feature

Re-open the case for #513. i.e. Support nvidia dali iterator as a possible data loader.

Pitch

I have submitted a wip pull request to the master. #789
I would like to know if it is ok for the test to be depending on nvidia-dali in addition to apex.

[smallzzy]

The dali iterator does not support resetting while epoch is not finished. I suppose I shall provide a warning for that.

[williamFalcon]

@smallzzy this is sick. super excited by this feature

[Borda]

@smallzzy we have cleared the API and now it should be stable... Mind resume your addition?

[brunoalano]

Some news about DALI support?

[Borda]

@brunoalano interested in implementing DALI?

[brunoalano]

@Borda Do you have a pipeline what should have be done to support it? But I'm available to do that with a minimal guidance (started using PyTorch Lightning and DALI these days)

[Borda]

we had almost done PR some time ago so I use you can resume from that point... #789

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[irustandi]

Is anyone working on this currently?

[irustandi]

I tried converting PyTorch Lightning's MNIST example to use DALI, it seems to work out of the box without requiring modification to the PyTorch Lightning's internals. See below.

https://gist.github.com/irustandi/3d180ed3ec9d7ff4e73d3fdbd67df3ca

[Borda]

@irustandi cool, mind add it as example, send a PR?

[irustandi]

@Borda sure, will create the PR.

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!

[ImMrMa]

I want to know that if dali with ddp and amp and dali's pipeline in distributed situation can work well? And have a perfect performance?

[Freed-Wu]

I want to know if pl.seed_everything() is valid for dali's seed?

[Borda]

Good question, I believe so... @awaelchli?

[awaelchli]

Lightning takes care of seeding the PyTorch related, built-in objects. That is, everything in the torch library, numpy, cuda and the dataloader workers. DALI and other libraries have their own logic regarding seeding and the user should consult their documentation regarding usage. pl.seed_everything does not cover these (because it can't).