Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VAD Train Problem #65

Open
Yjerry001 opened this issue Dec 25, 2024 · 1 comment
Open

VAD Train Problem #65

Yjerry001 opened this issue Dec 25, 2024 · 1 comment

Comments

@Yjerry001
Copy link

Yjerry001 commented Dec 25, 2024

In Step 2 "Prepare Bench2Drive data info" of the Prepare Bench2Drive Dataset, I reduced the maps due to memory limitations.
However, when running VAD Train
./adzoo/vad/dist_train.sh ./adzoo/vad/configs/VAD/VAD_base_e2e_b2d.py 1
I encountered the following error, which indicates that the dataset length is 0. Do you have any suggestions? Where should I check if the dataset was not successfully loaded? Thank you!

我在Prepare Bench2Drive Dataset的步骤2Prepare Bench2Drive data info中,由于内存原因,将地图减少了.
然后在运行VAD Train
./adzoo/vad/dist_train.sh ./adzoo/vad/configs/VAD/VAD_base_e2e_b2d.py 1
的过程中,遇到如下报错,显示dataset长度为0,请问有没有什么建议,dataset没有成功传入应该检查哪里,谢谢!

[rank0]: Traceback (most recent call last):
[rank0]:   File "./adzoo/vad/train.py", line 237, in <module>
[rank0]:     main()
[rank0]:   File "./adzoo/vad/train.py", line 226, in main
[rank0]:     custom_train_model(
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/adzoo/bevformer/mmdet3d_plugin/bevformer/apis/train.py", line 25, in custom_train_model
[rank0]:     custom_train_detector(
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/adzoo/bevformer/mmdet3d_plugin/bevformer/apis/mmdet_train.py", line 206, in custom_train_detector
[rank0]:     runner.run(data_loaders, cfg.workflow)
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/mmcv/runner/epoch_based_runner.py", line 129, in run
[rank0]:     epoch_runner(data_loaders[i], **kwargs)
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/mmcv/runner/epoch_based_runner.py", line 55, in train
[rank0]:     self.call_hook('after_train_epoch')
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/mmcv/runner/base_runner.py", line 297, in call_hook
[rank0]:     getattr(hook, fn_name)(self)
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/mmcv/runner/hooks/evaluation.py", line 266, in after_train_epoch
[rank0]:     self._do_evaluate(runner)
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/mmcv/core/evaluation/eval_hooks.py", line 121, in _do_evaluate
[rank0]:     results = self.test_fn(
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/adzoo/bevformer/apis/test.py", line 69, in custom_multi_gpu_test
[rank0]:     for i, data in enumerate(data_loader):
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 440, in __iter__
[rank0]:     return self._get_iterator()
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator
[rank0]:     return _MultiProcessingDataLoaderIter(self)
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1083, in __init__
[rank0]:     self._reset(loader, first_iter=True)
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1116, in _reset
[rank0]:     self._try_put_index()
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1350, in _try_put_index
[rank0]:     index = self._next_index()
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 620, in _next_index
[rank0]:     return next(self._sampler_iter)  # may raise StopIteration
[rank0]:   File "/home/yjr/.local/lib/python3.8/site-packages/torch/utils/data/sampler.py", line 288, in __iter__
[rank0]:     for idx in self.sampler:
[rank0]:   File "/home/yjr/Bench2DriveZoo/Bench2DriveZoo-uniad-vad/mmcv/datasets/samplers/distributed_sampler.py", line 32, in __iter__
[rank0]:     math.ceil(self.total_size / len(indices)))[:self.total_size]
[rank0]: ZeroDivisionError: division by zero
@jiaxiaosong1002
Copy link
Member

@Yjerry001 Not sure about the implications of reducing maps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants