[FilterVariantTranches] ValueError: fetch requires an index

[mumipyonpyon]

I am trying to use FilterVariantTranches in GATK 4.0.3.0 after running CNNScoreVariants

./gatk FilterVariantTranches \
    -V test.cnnscore.vcf \
    --snp-truth-vcf hapmap_3.3.hg19.sites.vcf \
    --indel-truth-vcf Mills_and_1000G_gold_standard.indels.hg19.sites.vcf \
    --info-key CNN_1D \
    --tranche 99.9 --tranche 99.0 --tranche 95 \
    --max-sites 8000 \
    -O test.cnnscore.filtered.vcf

There are also index files in the directory

test.cnnscore.vcf.idx (generated by CNNScoreVariants)
hapmap_3.3.hg19.sites.vcf.idx
Mills_and_1000G_gold_standard.indels.hg19.sites.vcf.idx

and I got the error

Traceback (most recent call last):
  File "/tmp/zzxzxzzxz/tranches.5887233932112211461.py", line 124, in <module>
    run()
  File "/tmp/zzxzxzzxz/tranches.5887233932112211461.py", line 10, in run
    write_tranches(args)
  File "/tmp/zzxzxzzxz/tranches.5887233932112211461.py", line 34, in write_tranches
    v_scored = allele_in_vcf(allele, variant, vcf_reader)
  File "/tmp/zzxzxzzxz/tranches.5887233932112211461.py", line 84, in allele_in_vcf
    variants = vcf_ram.fetch(variant.contig, variant.pos-1, variant.pos)
  File "pysam/libcbcf.pyx", line 4321, in pysam.libcbcf.VariantFile.fetch
ValueError: fetch requires an index

        at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
        at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:126)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeArgs(PythonScriptExecutor.java:170)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeScript(PythonScriptExecutor.java:151)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeScript(PythonScriptExecutor.java:121)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.FilterVariantTranches.doWork(FilterVariantTranches.java:134)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
        at org.broadinstitute.hellbender.Main.main(Main.java:289)

How can I fix this? Thank you!

[lucidtronix]

We are rewriting that tool in java to prevent issues like this. You could pull that branch from here: #4800.
Otherwise, I think pysam handles gzipped indexed vcfs better than plain text ones. So if you gzip your resource files with
bgzip -c hapmap_3.3.hg19.sites.vcf > hapmap_3.3.hg19.sites.vcf.gz
then index them with tabix:
tabix -p vcf hapmap_3.3.hg19.sites.vcf.gz
you may avoid this crash.

[mumipyonpyon]

Thank you very much!!

[cmnbroad]

Closing via #4800.