For Gigaspeech: "AssertionError: No recordings left after fixing the manifests."

[huangruizhe]

Hello, I was following this line in icefall to prepare gigaspeech manifests:

lhotse prepare gigaspeech --subset auto -j $nj \
    $dl_dir/GigaSpeech data/manifests

Then I got this error:

2023-03-02 02:07:33,633 INFO [gigaspeech.py:89] Processing GigaSpeech subset: DEV
Processing GigaSpeech JSON entries: 0it [00:00, ?it/s]2023-03-02 02:08:13,283 WARNING [qa.py:200] Trimmed 1 supervisions exceeding the end of the recording.
Processing GigaSpeech JSON entries: 1it [00:39, 39.70s/it]2023-03-02 02:08:13,460 WARNING [qa.py:200] Trimmed 1 supervisions exceeding the end of the recording.
Processing GigaSpeech JSON entries: 2it [00:39, 16.46s/it]2023-03-02 02:08:13,543 WARNING [qa.py:151] Removed 1 recordings with no corresponding supervisions.
Processing GigaSpeech JSON entries: 2it [00:39, 19.94s/it]
Traceback (most recent call last):
  File "/home/rhuang/mambaforge/envs/efrat/bin/lhotse", line 8, in <module>
    sys.exit(cli())
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/lhotse/bin/modes/recipes/gigaspeech.py", line 39, in gigaspeech
    prepare_gigaspeech(
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/lhotse/recipes/gigaspeech.py", line 113, in prepare_gigaspeech
    recordings, segments = fix_manifests(
  File "/home/rhuang/mambaforge/envs/efrat/lib/python3.8/site-packages/lhotse/qa.py", line 69, in fix_manifests
    assert (
AssertionError: No recordings left after fixing the manifests.

Would you suggest how to fix it? Or perhaps the gigaspeech download is incomplete? Thanks!

[huangruizhe]

Resolved.

I've validated the downloads following here, and I got:

utils/check_metadata_md5.sh: md5 verification failed for unknown version v0.2.4

So then I downloaded the latest version of Gigaspeech DEV and TEST sets and the script ran ok.
It seems Gigaspeech is evolving and can have changes here and there sometimes, even minor ones in DEV/TEST sets.