Hardware requirements

[mvisani]

Hi !

We've trying to develop a wrapper around Sirius CLI in Rust for our project. My problem is that the tests work on some machines but not on others.

On my machine (MB Pro 4 CPUs - 16GB memory) and on GitHub actions (Ubuntu, 4 CPUs - 16 GB memory) Sirius gets stuck here :

WARNING: <590>[TwoPhaseGibbsSampling-590] The graph seems to be badly connected. You might want to enforce more connections using local edge thresholds
STEP 1
STEP 2
STEP 3
STEP 4
heuristic: 9 / 28 ( 32.143 %)
PREPARING INNER EDGE SCORER TOOK 4 milliseconds
Sampling TOOK 291 milliseconds
number of badly connected compounds 8
number of compounds with more than 15 neighbours 0
Feb 12, 2024 4:26:19 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <590>[TwoPhaseGibbsSampling-590] The graph seems to be badly connected. You might want to enforce more connections using local edge thresholds
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <688>[FingerprintPreprocessingJJob-688 | [email protected]/z] Ignore fragmentation tree for C10H13N5O4 because it contains less than 3 vertices.
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <688>[FingerprintPreprocessingJJob-688 | [email protected]/z] Ignore fragmentation tree for C9H17NO8 because it contains less than 3 vertices.
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <688>[FingerprintPreprocessingJJob-688 | [email protected]/z] Ignore fragmentation tree for C8H18N3O5P because it contains less than 3 vertices.
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <688>[FingerprintPreprocessingJJob-688 | [email protected]/z] No suitable fragmentation tree left.
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <695>[FingerprintJJob-695 | [email protected]/z] No suitable input trees found.
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.jjobs.JJob lambda$logWarn$5
WARNING: <701>[FingerblastJJob-701 | [email protected]/z] No suitable input fingerprints found.
Feb 12, 2024 4:26:22 PM de.unijena.bioinf.rest.ProxyManager decorateWithPoolSettings
INFO: Starting http Client with MaxPerRoute=1 / maxTotal=1 (CPU-Threads=2).
Feb 12, 2024 4:26:23 PM de.unijena.bioinf.rest.ProxyManager decorateWithPoolSettings
INFO: Starting http Client with MaxPerRoute=1 / maxTotal=1 (CPU-Threads=2).

After running this command :

sirius -i tests/data/input_sirius.mgf --output tests/data/output_sirius_default --maxmz=800.0 formula zodiac fingerprint structure canopus write-summaries

On other machines (also MacBook Pro and Linux) with more CPUs or RAM, Sirius works like a charm. We've been using the same input file, the same logins, and the same parameters.

I was wondering then if this could be a hardware requirement problem.

Thanks in advance !

[mfleisch]

Hey,
in general 4 CPUs - 16GB memory) should be sufficient. However memory consumption of zodiac increases quadratically with the number of features. So in case you think it might be an memory issue you could check whether it runs without zodiac. However from the log it does not look memory related but rather like a network or multi threading problem.

Are you already using the most recent release (SIRIUS v5.8.6)? It contains a few improvements regarding how remote and local steps are parallelized.

You could also try to manually increase the number of threads (--cores) SIRIUS is allowed to use to check whether it is related to multithreading.

[mfleisch]

It might also be of interest for you that SIRIUS 6 (major release) is coming soon, which will change some parts the command line tool as well es how projects are stored. Writing summary files will still be possible though.

However, SIRIUS 6 will provide an improved and stable API. It is intended as interface for external tools and backwards compatibility has high priority for it. It is based on the OpenAPI spec which allows to auto-generate SDKs for various languages including rust: https://openapi-generator.tech/docs/generators/rust.

Implementing your wrapper based on the API might be nicer to implement, more efficient and more stable regarding future SIRIUS releases then using the CLI.

In case you are interested and might be able to contribute some rust knowledge we might even be able to add a rust SDK to our officially provided SDKs. https://github.com/boecker-lab/sirius-client-openAPI

Fee free to contact me in case you want to discuss this further.

[mvisani]

Hey, thanks for the reply.

We are indeed working with the latest version of SIRIUS. I've tested your suggestion and specified the number of cores, and it works like a charm. Thank you! However, this brings up another question: why does sirius --help indicate that all cores are being used by default, yet when I don't specify that parameter, it doesn't work? As you mentioned, this seems to be a multithreading problem.

Regarding your second comment, I completely agree that implementing the wrapper around the API would make much more sense. Developing a Rust SDK would certainly be a nice challenge, but I'll need to confer with @LucaCappelletti94 and @oolonek to see how we could tackle this. Out of curiosity, when do you expect to release SIRIUS 6?

[mfleisch]

SIRIUS 6 is released and API is stable but probably too late? In case a rust client is still interesting for you I am happy to provide support. I am closing this since this is not about the initial topic of this issue. Feel free joint our Gitter community https://matrix.to/#/#sirius-ms:gitter.im to talk about the API or a potential Rust SDK.