Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unable to make Sirius use Gurobi solver #147

Open
flass opened this issue Jan 23, 2024 · 1 comment
Open

unable to make Sirius use Gurobi solver #147

flass opened this issue Jan 23, 2024 · 1 comment
Labels
Selected for Development Mark if this issue has been linked to an internal development issue. under-investigation

Comments

@flass
Copy link

flass commented Jan 23, 2024

Hi,

I have been trying (quite hard) to make Sirus use Gurobi as the iLP solver instead of the built-in CLP, but so far to no avail.

Could you please help me see what I'm doing wrong?

I was running Sirius 5.8.5 using the executable dowloaded from this repo (https://github.com/boecker-lab/sirius/releases/download/v5.8.5/sirius-5.8.5-linux64.zip), with the following environment variables set:

GRB_LICENSE_FILE=/software/gurobi/gurobi.lic
GUROBI_HOME=/software/gurobi/gurobi911/linux64
LD_LIBRARY_PATH=/software/gurobi/gurobi911/linux64/lib:/software/lsf-farm5/10.1/linux3.10-glibc2.17-x86_64/lib:/software/gurobi/gurobi911/linux64/lib:/usr/lib/oracle/12.1/client64/lib

just showing that these point to relevant files/locations:

[fl4@farm5-head1 sirius_Test]$ ls -l /software/gurobi/gurobi.lic
-rw-r--r-- 1 root root 951 Nov 17 15:31 /software/gurobi/gurobi.lic
[fl4@farm5-head1 sirius_Test]$ ls -l /software/gurobi/gurobi911/linux64/
total 960
drwxr-xr-x  2 root root    211 Dec 15  2020 bin
drwxr-xr-x  8 root root    446 Dec 15  2020 docs
-rw-r--r--  1 root root 162413 Dec 15  2020 EULA.pdf
drwxr-xr-x 12 root root    214 Dec 15  2020 examples
drwxr-xr-x  3 root root     85 Dec 15  2020 include
drwxr-xr-x  9 root root    680 Dec 15  2020 lib
drwxr-xr-x  3 root root    548 Dec 15  2020 matlab
drwxr-xr-x  2 root root     69 Dec 15  2020 R
-rw-r--r--  1 root root   8825 Dec 15  2020 ReleaseNotes.html
-rw-r--r--  1 root root   2840 Dec 15  2020 setup.py
drwxr-xr-x  4 root root     68 Dec 15  2020 src
[fl4@farm5-head1 sirius_Test]$ ls -l /software/gurobi/gurobi911/linux64/lib/
total 44336
-rw-r--r--  1 root root   145920 Dec 15  2020 gurobi91.netstandard20.dll
-rw-r--r--  1 root root   450695 Dec 15  2020 gurobi91.netstandard20.xml
-rw-r--r--  1 root root    86293 Dec 15  2020 gurobi.jar
-rw-r--r--  1 root root   159945 Dec 15  2020 gurobi-javadoc.jar
-rwxr-xr-x  1 root root      769 Dec 15  2020 gurobi.py
-rwxr-xr-x  1 root root 38305304 Dec 15  2020 libgurobi91_light.so
lrwxrwxrwx  1 root root       20 Dec 15  2020 libgurobi91.so -> ./libgurobi.so.9.1.1
lrwxrwxrwx  1 root root       20 Dec 15  2020 libgurobi_c++.a -> ./libgurobi_g++4.2.a
-rw-r--r--  1 root root   615816 Dec 15  2020 libgurobi_g++4.2.a
-rw-r--r--  1 root root   564336 Dec 15  2020 libgurobi_g++5.2.a
-rwxr-xr-x  1 root root   112203 Dec 15  2020 libGurobiJni91.so
-rwxr-xr-x  1 root root 42291983 Dec 15  2020 libgurobi.so.9.1.1
drwxr-xr-x  3 root root       55 Dec 15  2020 python2.7_utf16
drwxr-xr-x  3 root root       55 Dec 15  2020 python2.7_utf32
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.6_utf32
drwxr-xr-x 12 root root     5190 Dec 15  2020 python3.7
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.7_utf32
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.8_utf32
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.9_utf32
-rwxr-xr-x  1 root root     1215 Dec 15  2020 rootcert.pem

here is the command I ran:

sirius/bin/sirius -i filtered_output_sirius.mgf formula -p orbitrap --ilp-solver gurobi

and this is the header of the standard output that I get:

Jan 08, 2024 9:24:24 AM org.apache.commons.beanutils.FluentPropertyBeanIntrospector introspect
INFO: Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Sirius Workspace Successfull initialized at: /nfs/users/nfs_f/fl4/.sirius-5.8
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: You run SIRIUS 5.8.5 on linux_amd64
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: You run SIRIUS in 'CLI' mode.
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Sirius was compiled with the following ILP solvers: GLPK-v1.7.0 (included), Gurobi-v9.1.1, CPLEX-v12.7.1, COIN-OR-v2.10.8.4
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Treebuilder priorities loaded from 'sirius.properties' are: [GUROBI]
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: CPU check done. 1 cores that handle 1 threads were found.
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ChemistryBase.jobs.SiriusJobs getGlobalJobManager
INFO: Job manager successful initialized with 1 CPU thread(s) and 1 IO thread(s).
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.rest.ProxyManager decorateWithPoolSettings
INFO: Starting http Client with MaxPerRoute=1 / maxTotal=5 (CPU-Threads=1).
Jan 08, 2024 9:24:27 AM de.unijena.bioinf.auth.AuthService getToken
WARNING: No login Found: Not Logged in, No valid refresh token Available
Jan 08, 2024 9:24:27 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Web API initialized.
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.ms.frontend.Run parseArgs
INFO: Running with following arguments: [-i, test_dataset_Claudio/filtered_output_sirius.mgf, formula, -p, orbitrap, --ilp-solver, gurobi]
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <0>[InputExpanderJJob-0] Running post processing
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <0>[InputExpanderJJob-0] Running cleanup
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logInfo$9
INFO: <1>[InputExpanderJJob-1] Expanding Input Files: 'test_dataset_Claudio/filtered_output_sirius.mgf'...
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logInfo$9
INFO: <1>[InputExpanderJJob-1] ...Input Files successfully expanded!
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <1>[InputExpanderJJob-1] Running post processing
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <1>[InputExpanderJJob-1] Running cleanup
Jan 08, 2024 9:24:29 AM de.unijena.bioinf.ms.frontend.subtools.sirius.SiriusOptions setSolver
INFO: ILP solver changed to 'GUROBI' by command line.
...

and this is the error I get after a while (reiterated for all the compounds that are being assessed - Sirius does not crash, it just caries on failing like this):

Jan 08, 2024 2:43:35 PM de.unijena.bioinf.FragmentationTreeConstruction.computation.tree.TreeBuilderFactory getTreeBuilderFromClass
WARNING: Could not load GrbSolver! Sirius was compiled with the following ILP solvers: GLPK-v1.7.0 (included), Gurobi-v9.1.1, CPLEX-v12.7.1, COIN-OR-v2.10.8.4: Could not initialize class de.unijena.bioinf.FragmentationTreeConstruction.com
putation.tree.ilp.GrbSolver
Jan 08, 2024 2:43:35 PM de.unijena.bioinf.FragmentationTreeConstruction.computation.tree.TreeBuilderFactory getTreeBuilder
SEVERE: Your system does not ship with any instantiatable ILP solver. Please install either CLP,  Gurobi or CPLEX to use SIRIUS.
Jan 08, 2024 2:43:36 PM de.unijena.bioinf.jjobs.JJob lambda$logError$2

when terminating the job manually, I got this last error (which I suspect to be just due to user interuption, but mentioning it just in case):

SEVERE: <1265504>[RecalibrationJob-1265504] Failed!
java.lang.NullPointerException: Cannot invoke "de.unijena.bioinf.FragmentationTreeConstruction.computation.tree.TreeBuilder.computeTree()" because "finalBuilder" is null
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.recalibrate(FasterTreeComputationInstance.java:577)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance$RecalibrationJob.compute(FasterTreeComputationInstance.java:549)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance$RecalibrationJob.compute(FasterTreeComputationInstance.java:535)
        at de.unijena.bioinf.jjobs.BasicJJob.call(BasicJJob.java:117)
        at de.unijena.bioinf.jjobs.BasicMasterJJob$1.compute(BasicMasterJJob.java:101)
        at java.base/java.util.concurrent.RecursiveTask.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.awaitDone(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.join(Unknown Source)
        at de.unijena.bioinf.jjobs.BasicJJob.awaitResult(BasicJJob.java:211)
        at de.unijena.bioinf.jjobs.JJob.takeResult(JJob.java:249)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.estimateTreeSizeAndRecalibration(FasterTreeComputationInstance.java:307)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.compute(FasterTreeComputationInstance.java:173)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.compute(FasterTreeComputationInstance.java:55)
        at de.unijena.bioinf.jjobs.BasicJJob.call(BasicJJob.java:117)
        at de.unijena.bioinf.jjobs.BasicMasterJJob$1.compute(BasicMasterJJob.java:101)
        at java.base/java.util.concurrent.RecursiveTask.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.awaitDone(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.join(Unknown Source)
        at de.unijena.bioinf.jjobs.BasicJJob.awaitResult(BasicJJob.java:211)
        at de.unijena.bioinf.sirius.Sirius$SiriusIdentificationJob.compute(Sirius.java:640)
        at de.unijena.bioinf.sirius.Sirius$SiriusIdentificationJob.compute(Sirius.java:623)
        at de.unijena.bioinf.jjobs.BasicJJob.call(BasicJJob.java:117)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

Can you help me with this please?

Many thanks,

Florent Lassalle
Principal Bioinformatician
PaM informatics
Parasites and Microbes Programme
Wellcome Sanger Institute
@mfleisch mfleisch added under-investigation Selected for Development Mark if this issue has been linked to an internal development issue. labels Sep 4, 2024
@mfleisch
Copy link
Contributor

mfleisch commented Sep 4, 2024

Hey
we are currently checking this issue for SIRIUS 6 but we are likely not looking in to it for SIRIUS 5 anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Selected for Development Mark if this issue has been linked to an internal development issue. under-investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@flass @mfleisch