Add verbose mode for dataset constructors to print directory scan stats + add a warning when video loading fails

[vadimkantorov]

I'm trying to create VideoClips object with my custom folder with a video. It returns zero results, because it can't read_video_timestamps (under PyAV backend) with an error av.error.InvalidDataError: [Errno 1094995529] Invalid data found when processing input: 'data/pseudo-kinetics/train_256/class0/P01_01.MP4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found

Maybe it's well an invalid file, but it's indeed better to print a warning

vision/torchvision/io/video.py

Line 408 in 7536e29

# TODO add a warning

(at least when some verbose flag equals True - maybe worth introducing verbose flag in dataset constructors) and maybe print the stats over all files (how many loaded, how many skipped because of extensions, how many had errors while loading) - this would save a lot of time when creating a new dataset and that has some problems

cc @bjuncek

[datumbox]

Thanks for reporting.

I think it would be good to add a warning there instead of silently catching the exception. If you are interested, please send a PR that introduces a warning.

[vadimkantorov]

File lists became available: https://github.com/cvdfoundation/kinetics-dataset

So extensions could now be set properly

[bjuncek]

@vadimkantorov with #3680 we introduce a new kinetics dataset class (Kinetics(num_classes=400)) that should solve the issue with different extension (and you can also supply your own list of extensions btw). #3932 raises an error for invalid files; #3961 raises the warning for the possibly corrupt files.

As for the dataset verbose dataset statistics, that is something that we can perhaps keep in a separate issue in order discuss with @pmeier after the dataset revamp has been implemented?