Problem running EMU with some samples

[AlexandreThibodeauUdM]

Hello! I want to stress out how EMU is the best tool for now for Nanopore 16S full sequencing! I am using it on a server and 90% of samples get through. But for some samples, (some with lots of sequences, some with very few sequences), it does not work. I get the following error message:

traceback (most recent call last):
File "/home/alexthi/emu/emu", line 804, in
SAM_FILE = generate_alignments(args.input_file, out_file, args.db)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alexthi/emu/emu", line 416, in generate_alignments
subprocess.check_output("minimap2 -ax {} -t {} -N {} -p .9 -K {} {} {} -o {}".
File "/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'minimap2 -ax map-ont -t 7 -N 50 -p .9 -K 100000000 /scratch/alexthi/EMUdatabase/species_taxid.fasta vitfaniebarcode92chop.fastq -o /scratch/alexthi/nanoconcat/emuresults/vitfaniebarcode92chop_emu_alignments.sam' died with <Signals.SIGKILL: 9>.

Any clue of what might be causing that?

I have tried to change -K, number of CPU, memory allocated, etc, but still get the same error.

[kdc10]

<Signals.SIGKILL: 9> typically means the minimap2 run could not complete due to lack of memory space.

Which version of emu and which version of minimap2 are you running?

[AlexandreThibodeauUdM]

Thank you for answering. Emu, it is version 3.5.0. As for miniap2, it is version 2.28.

[AlexandreThibodeauUdM]

So I upgraded Emu version. This is what I run on the server.

#!/bin/bash

#SBATCH --time=11:59:00
#SBATCH --account=def-alexthi
#SBATCH --mem=2500000M
#SBATCH --mail-user=[email protected]
#SBATCH --cpus-per-task=32
#SBATCH --mail-type=BEGIN
#SBATCH --mail-type=END
#SBATCH --mail-type=FAIL
#SBATCH --mail-type=REQUEUE
#SBATCH --mail-type=ALL
#SBATCH --output=nanopore.out

python ~/emu/emu abundance --keep-counts --min-abundance 0.000001 --db $SCRATCH/EMUdatabase/ --output-dir $SCRATCH/nanoconcat/emuresults --threads $SLURM_CPUS_PER_TASK vitfaniebarcode04chop.fastq

many thanks!

[jodjo86]

Bonjour Alexandre,
It's always nice to meet another Québécois :) It looks like you're doing your analysis on a Compute Canada (CC) cluster SLURM. I see two options:

1. As Kristen said, it's possible that its due by a lack of memory space. I'm not sure how memory allocation is done with CC cluster. It's possible that memory is divided by the number of CPUs. I'll do a test with 16 cores instead of 32. I've done a few speed tests here #33.

2. It's also possible that the error is caused by a timeout (SLURM kills processes after a certain time). Have you tried running just one task at a time? How long does a single task take to run?