Skip to content

Commit

Permalink
Updated the blastn runtime requirements to avoid basement jobs
Browse files Browse the repository at this point in the history
Indeed, 85.9% of jobs complete within 4 hours, 98% within 47 hours, and 99.7% within 167 hours
  • Loading branch information
muffato committed Oct 14, 2024
1 parent abe2b76 commit dfb4655
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 6 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ The pipeline is now considered to be a complete and suitable replacement for the
"grid plots".
- Fill in accurate read information in the blobDir. Users are now reqiured
to indicate in the samplesheet whether the reads are paired or single.
- Updated the Blastn settings to allow 7 days runtime at most, since that
covers 99.7% of the jobs.

### Software dependencies

Expand Down
13 changes: 7 additions & 6 deletions conf/base.config
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,15 @@ process {

withName: "BLAST_BLASTN" {

// There are blast failures we don't know how to fix. Just ignore for now
errorStrategy = { task.exitStatus in ((130..145) + 104) ? (task.attempt == process.maxRetries ? 'ignore' : 'retry') : 'finish' }
// There are blast failures we don't know how to fix. We just give up after 3 attempts
errorStrategy = { task.exitStatus in ((130..145) + 104) ? (task.attempt == 3 ? 'ignore' : 'retry') : 'finish' }


// Most jobs complete quickly but some need a lot longer. For those outliers,
// the CPU usage remains usually low, often nearing a single CPU
cpus = { check_max( 6 - (task.attempt-1), 'cpus' ) }
memory = { check_max( 1.GB * Math.pow(4, task.attempt-1), 'memory' ) }
time = { check_max( 10.h * Math.pow(4, task.attempt-1), 'time' ) }
// the CPU usage remains usually low, averaging a single CPU
cpus = { check_max( task.attempt == 1 ? 4 : 1, 'cpus' ) }
memory = { check_max( 2.GB, 'memory' ) }
time = { check_max( task.attempt == 1 ? 4.h : ( task.attempt == 2 ? 47.h : 167.h ), 'time' ) }
}

withName:CUSTOM_DUMPSOFTWAREVERSIONS {
Expand Down

0 comments on commit dfb4655

Please sign in to comment.