core dump

[linzhi2013]

Hi there,

Thanks for the tool!

When I tried the meshclust 3.0, I got the core dump error, do you have any suggestions for this? thank you!

The compute-node of the cluster has 56 cores (112 threads), 1.5T RAM, and we did not limit how much RAM the meshclust would like to use.

Best
Guanliang

-rw-rw-r-- 1 gmeng 1.5G Jun 13 17:15 combined.fa
-rw-rw-r-- 1 gmeng  112 Jun 14 10:00 meshclust3.sh
-rw-r--r-- 1 gmeng 5.9K Jun 15 20:16 meshclust3.sh.o539214
-rw------- 1 gmeng  18G Jun 15 22:18 core.229599
-rw-r--r-- 1 gmeng  416 Jun 15 22:18 meshclust3.sh.e539214

$ grep -c '>' combined.fa
5652580

meshclust3.sh:

/home/gmeng/soft/MeShClust_v3/Identity/bin/meshclust -d combined.fa -t 0.6  -o out.clstr -c 80 -e y -a n -p 10

meshclust3.sh.o539214:

MeShClust v3.0 is developed by Hani Z. Girgis, PhD.

This program clusters DNA sequences using identity scores obtained without alignment.

Copyright (C) 2021-2022 Hani Z. Girgis, PhD

Academic use: Affero General Public License version 1.

Any restrictions to use for profit or non-academics: Alternative commercial license is required.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Please contact Dr. Hani Z. Girgis ([email protected]) if you need more information.

Please cite the following papers:
	1. Identity: Rapid alignment-free prediction of sequence alignment identity scores using
	self-supervised general linear models. Hani Z. Girgis, Benjamin T. James, and Brian B.
	Luczak. NAR GAB, 3(1):lqab001, 2021.
	2. MeShClust: an intelligent tool for clustering DNA sequences. Benjamin T. James,
	Brian B. Luczak, and Hani Z. Girgis. Nucleic Acids Res, 46(14):e83, 2018.
	3. MeShClust v3.0: High-quality clustering of DNA sequences using the mean shift algorithm
	and alignment-free identity scores. Hani Z. Girgis. A great journal. 2022.

Database file: combined.fa
Output file: out.clstr
Cores: 80
Provided threshold: 0.6
Block size for all vs. all: 25000
Block size for reading sequences: 100000
Number of data passes: 10
Can assign all: No


Average: 756
K: 4
Histogram size: 256
A histogram entry is 16 bits.
Generating data.
Preparing data ...
	Positive examples: 10000
	Training size: 5000
	Validation size: 5000
Better performance of: 0.00324074
	chi_squared x jeffrey_divergence
Better performance of: 0.00278104
	chi_squared x jeffrey_divergence
	chi_squared^2 x d2_s_r^2
Better performance of: 0.00275123
	chi_squared x jeffrey_divergence
	chi_squared^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
Better performance of: 0.00271437
	chi_squared x jeffrey_divergence
	chi_squared^2 x d2_s_r^2
	bray_curtis^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
Better performance of: 0.00266334
	chi_squared x squared_chord
	chi_squared x jeffrey_divergence
	chi_squared^2 x d2_s_r^2
	bray_curtis^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
	kulczynski_2^2 x d2_s_r^2
Better performance of: 0.00263148
	squared_chord
	chi_squared x squared_chord
	chi_squared x jeffrey_divergence
	chi_squared^2 x d2_s_r^2
	bray_curtis^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
	kulczynski_2^2 x d2_s_r^2
Better performance of: 0.00257594
	squared_chord
	chi_squared x squared_chord
	chi_squared x jeffrey_divergence
	hellinger x hellinger^2
	chi_squared^2 x d2_s_r^2
	bray_curtis^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
	kulczynski_2^2 x d2_s_r^2
Better performance of: 0.00249854
	squared_chord
	manhattan x simMM
	chi_squared x squared_chord
	chi_squared x jeffrey_divergence
	hellinger x hellinger^2
	chi_squared^2 x d2_s_r^2
	bray_curtis^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
	kulczynski_2^2 x d2_s_r^2
Selected statistics:
	squared_chord
	manhattan x simMM
	chi_squared x squared_chord
	chi_squared x jeffrey_divergence
	hellinger x hellinger^2
	chi_squared^2 x d2_s_r^2
	bray_curtis^2 x d2_s_r^2
	squared_chord^2 x hellinger^2
	kulczynski_2^2 x d2_s_r^2
Finished training.
	MAE: 0.036734
	MSE: 0.00249854
Optimizing ...
Validating ...
	MAE: 0.0426102
	MSE: 0.00325363

Clustering ...

Data run 1 ...
	Processed sequences: 25000
	Unprocessed sequences: 0
	Found centers: 772
	Processed sequences: 50000
	Unprocessed sequences: 24657
	Found centers: 770
	Processed sequences: 100478
	Unprocessed sequences: 41448
	Found centers: 1278
	Processed sequences: 166024
	Unprocessed sequences: 32518
	Found centers: 2628
	Processed sequences: 206655
	Unprocessed sequences: 27580
	Found centers: 3034
	Processed sequences: 338846
	Unprocessed sequences: 65658
	Found centers: 3620
	Processed sequences: 348903
	Unprocessed sequences: 50307
	Found centers: 4308
	Processed sequences: 414183
	Unprocessed sequences: 67888
	Found centers: 4653
	Processed sequences: 428889
	Unprocessed sequences: 56801
	Found centers: 5147
	Processed sequences: 473924
	Unprocessed sequences: 66571
	Found centers: 5560
	Processed sequences: 591912
	Unprocessed sequences: 101368
	Found centers: 6457
	Processed sequences: 599863
	Unprocessed sequences: 83946
	Found centers: 6943
	Processed sequences: 682732
	Unprocessed sequences: 112078
	Found centers: 7277
	Processed sequences: 694499
	Unprocessed sequences: 97930
	Found centers: 7757
	Processed sequences: 752209
	Unprocessed sequences: 114752
	Found centers: 8067
	Processed sequences: 767163
	Unprocessed sequences: 94407
	Found centers: 8447
	Processed sequences: 867163
	Unprocessed sequences: 141679
	Found centers: 8792
	Processed sequences: 875812
	Unprocessed sequences: 125026
	Found centers: 9248
	Processed sequences: 950986
	Unprocessed sequences: 155363
	Found centers: 9586
	Processed sequences: 962281
	Unprocessed sequences: 137454
	Found centers: 10001
	Processed sequences: 1050620
	Unprocessed sequences: 173768
	Found centers: 10430
	Processed sequences: 1060816
	Unprocessed sequences: 156809
	Found centers: 10884
	Processed sequences: 1138833
	Unprocessed sequences: 189905
	Found centers: 11240
	Processed sequences: 1219898
	Unprocessed sequences: 191996
	Found centers: 12162
	Processed sequences: 1234377
	Unprocessed sequences: 173682
	Found centers: 12615
	Processed sequences: 1328038
	Unprocessed sequences: 210768
	Found centers: 13095
	Processed sequences: 1338108
	Unprocessed sequences: 194114
	Found centers: 13563
	Processed sequences: 1413309
	Unprocessed sequences: 217638
	Found centers: 13916
	Processed sequences: 1426200
	Unprocessed sequences: 203726
	Found centers: 14366
	Processed sequences: 1482720
	Unprocessed sequences: 217439
	Found centers: 14648
	Processed sequences: 1549592
	Unprocessed sequences: 216905
	Found centers: 15453
	Processed sequences: 1566431
	Unprocessed sequences: 205939
	Found centers: 15909
	Processed sequences: 1610994
	Unprocessed sequences: 211989
	Found centers: 16228

meshclust3.sh.e539214:

Mean 1 (mean1) and Mean 2 (mean2) cannot be zeros. Mean 1 is: 0, mean 2 is: 0.226562

terminate called after throwing an instance of 'std::exception'
  what():  std::exception
/opt/gridengine/default/spool/compute-0-0/job_scripts/539214: Zeile 1: 229599 Abgebrochen             (Speicherabzug geschrieben) /home/gmeng/soft/MeShClust_v3/Identity/bin/meshclust -d combined.fa -t 0.6 -o out.clstr -c 80 -e y -a n -p 10

[hani-girgis]

Hi, Guanliang.

Thanks for your interest in MeShClust.

I suspect that one of the sequences in combined.fa is too short or has many uncertain nucleotides, e.g., N. Can you please verify and let me know?

Best regards.

Hani Z. Girgis, PhD