tools that expect unaligned reads shouldn't validate the sequence dictionary

[lbergelson]

Tools that take an unaligned bam shouldn't expect that the bam has contigs that match the reference.

These include BwaAndMarkDuplicatesSpark, BwaSpark, and ReadsPipelineSpark (when running in alignment mode.)

[nyl2002]

Are the errors below part of this, when starting BwaSpark with spark-submit?
I activated "--disable-sequence-dictionary-validation true", but that doesn't help.

It is very unclear, why a BAM is not recognized as a BAM file. I have tried all kinds of ways to make sure that it is a BAM and not a SAM file.
The documentation for BwaSpark also says "BAM/SAM/CRAM file containing reads", so if SAM files are really not possible, that should probably be changed.
...
Even on verbosity DEBUG, the comments are not at all helpful to get at the problem.
E.g. "Cannot retrieve file pointers within SAM text files."
Is that a general statement about SAM files? Or does it only say, that in this specific SAM file (which is actually a BAM file), file pointers cannot be found?
What pointers are meant exactly?
How could this be fixed?

"SamReaderFactory	Unable to detect file format from input URL or stream, assuming SAM format."
Which URL?
Which stream?
Why would this happen? What could be the error?
The SAM/BAM distinction seems very unclear. It would be more helpful, if some specific missing aspect (e.g. not queryname sorted) would be clearly declared as the culprit.
...
00:29 DEBUG: [kryo] Write: SAMFileHeader{VN=1.5, SO=queryname}
...
WARNING	2018-01-16 02:11:25	SamReaderFactory	Unable to detect file format from input URL or stream, assuming SAM format.
...
java.lang.UnsupportedOperationException: Cannot retrieve file pointers within SAM text files.
	at htsjdk.samtools.SAMTextReader.getFilePointerSpanningReads(SAMTextReader.java:185)
...

[lbergelson]

@nyl2002 What's your command line that's hitting problems? Are you trying to run BWA-MEM spark on a SAM file or on a BAM file?

I agree that we should change documentation and produce a better error message if it's failing on SAM files.