very-sensitive-local consistency between 2.2.9/2.3.0/2.3.1

[martinghunt]

Is it possible to make 2.3.1 produce the same results as 2.2.9 with the --very-sensitive-local option? I've seen the comment in the release notes about --score-min.

I'm testing this without success on small test data:

$ cat read1.fq
@read/1
CCTCCACAGGATGGTGGTATACCTG
+
IIIIIIIIIIIIIIIIIIIIIIIII
$ cat read2.fq
@read/2
TCTGCGCTTAGCTAAGGTGGATGAA
+
IIIIIIIIIIIIIIIIIIIIIIIII
$ cat ref.fa
>ref
TCCACAGGATGGTGGTATACCTGAGGCCAAAGGATACAGATCTTGTGGGAAAGGTCCGCC
GGGAAGTCCTTCTGTCGTGCTTTTTATCGGGTCTGTACAAGGACGGATGGTTTCCGGCAT
ACCATAATGTTCTTAGGGCTTACCATAGAGGTACACTAAAAAGTGTTTCATCCACCTTAG
CTAAGCGCAG

With 2.2.9 and --very-sensitive-local, both reads are mapped. Perfect matches except 1 or 2 bases soft-clipped:

$ ~/bin/bowtie2-2.2.9/bowtie2 -1 read1.fq -2 read2.fq -x ref.fa --very-sensitive-local
read 99 ref 1 44 2S23M = 167 192 CCTCCACAGGATGGTGGTATACCTG IIIIIIIIIIIIIIIIIIIIIIIII AS:i:46XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:23 YS:i:48 YT:Z:CP
read 147 ref 167 44 24M1S = 1 -192 TTCATCCACCTTAGCTAAGCGCAGA IIIIIIIIIIIIIIIIIIIIIIIII AS:i:48XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:24 YS:i:46 YT:Z:CP

Same result with 2.3.0. But with 2.3.1 the reads are not mapped:

$ ~/bin/bowtie2-2.3.1-legacy/bowtie2 -1 read1.fq -2 read2.fq -x ref.fa --very-sensitive-local
read 77 * 0 0 * * 0 0 CCTCCACAGGATGGTGGTATACCTG IIIIIIIIIIIIIIIIIIIIIIIII YT:Z:UP
read 141 * 0 0 * * 0 0 TCTGCGCTTAGCTAAGGTGGATGAA IIIIIIIIIIIIIIIIIIIIIIIII YT:Z:UP

If I read the help correctly, the option --very-sensitive-local is the same as doing this (and at least the results are the same):

$ ~/bin/bowtie2-2.2.9/bowtie2 -1 read1.fq -2 read2.fq -x ref.fa -D 20 -R 3 -N 0 -L 20 -i S,1,0.50 --score-min G,0,10 --local
read 99 ref 1 44 2S23M = 167 192 CCTCCACAGGATGGTGGTATACCTG IIIIIIIIIIIIIIIIIIIIIIIII AS:i:46XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:23 YS:i:48 YT:Z:CP
read 147 ref 167 44 24M1S = 1 -192 TTCATCCACCTTAGCTAAGCGCAGA IIIIIIIIIIIIIIIIIIIIIIIII AS:i:48XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:24 YS:i:46 YT:Z:CP

Using the same options with 2.3.0 or 2.3.1 gives this error:

Error: the match penalty is greater than 0 (2) but the --score-min function can be less than or equal to zero.  Either let the match penalty be 0 or make --score-min always positive.
Error: Encountered internal Bowtie 2 exception (#1)
Command: /nfs/users/nfs_m/mh12/bin/bowtie2-2.3.1-legacy/bowtie2-align-s --wrapper basic-0 -x ref.fa -D 20 -R 3 -N 0 -L 20 -i S,1,0.50 --score-min G,0,10 --local -1 read1.fq -2 read2.fq
(ERR): bowtie2-align exited with value 1

I've tried various things with no luck. Is it possible to get 2.3.0 and 2.3.1 to behave consistently with 2.2.9 --very-sensitive-local using the right combination of options?

[BenLangmead]

As you can see, we tried to make it harder for people to set the minimum-score function in a way that was not consistent with the other components of the scoring scheme. In this case, the specific fact that activates the error message is that the specified function evaluates to 0 give a read of length 1.

Here's where we do the check

bowtie2/simple_func.h

Line 85 in 89203e5

bool alwaysPositive() const {

Maybe that's a little more aggressive than we need to be. If we checked at length=2, for example, rather than length=1, then we would not get the error. The question is: is the need for consistency across versions compelling enough for us to make such a change?

[martinghunt]

I need consistency across versions. In my code I can detect the bowtie2 version and use the appropriate options depending on the version. Is there a combination of options I can use with 2.3.1 to make it sensitive enough to map those reads? I can't get 2.3.1 to map them at all.

[BenLangmead]

Can you try --score-min G,1,10 (same as original option but changing the middle 0 to a 1)?

Separately, we have an idea for how to restore the --score-min G,0,10 functionality in the next version of Bowtie 2.

[martinghunt]

Brilliant, thanks! Works perfectly.