unable to make Sirius use Gurobi solver

[flass]

Hi,

I have been trying (quite hard) to make Sirus use Gurobi as the iLP solver instead of the built-in CLP, but so far to no avail.

Could you please help me see what I'm doing wrong?

I was running Sirius 5.8.5 using the executable dowloaded from this repo (https://github.com/boecker-lab/sirius/releases/download/v5.8.5/sirius-5.8.5-linux64.zip), with the following environment variables set:

GRB_LICENSE_FILE=/software/gurobi/gurobi.lic
GUROBI_HOME=/software/gurobi/gurobi911/linux64
LD_LIBRARY_PATH=/software/gurobi/gurobi911/linux64/lib:/software/lsf-farm5/10.1/linux3.10-glibc2.17-x86_64/lib:/software/gurobi/gurobi911/linux64/lib:/usr/lib/oracle/12.1/client64/lib

just showing that these point to relevant files/locations:

[fl4@farm5-head1 sirius_Test]$ ls -l /software/gurobi/gurobi.lic
-rw-r--r-- 1 root root 951 Nov 17 15:31 /software/gurobi/gurobi.lic
[fl4@farm5-head1 sirius_Test]$ ls -l /software/gurobi/gurobi911/linux64/
total 960
drwxr-xr-x  2 root root    211 Dec 15  2020 bin
drwxr-xr-x  8 root root    446 Dec 15  2020 docs
-rw-r--r--  1 root root 162413 Dec 15  2020 EULA.pdf
drwxr-xr-x 12 root root    214 Dec 15  2020 examples
drwxr-xr-x  3 root root     85 Dec 15  2020 include
drwxr-xr-x  9 root root    680 Dec 15  2020 lib
drwxr-xr-x  3 root root    548 Dec 15  2020 matlab
drwxr-xr-x  2 root root     69 Dec 15  2020 R
-rw-r--r--  1 root root   8825 Dec 15  2020 ReleaseNotes.html
-rw-r--r--  1 root root   2840 Dec 15  2020 setup.py
drwxr-xr-x  4 root root     68 Dec 15  2020 src
[fl4@farm5-head1 sirius_Test]$ ls -l /software/gurobi/gurobi911/linux64/lib/
total 44336
-rw-r--r--  1 root root   145920 Dec 15  2020 gurobi91.netstandard20.dll
-rw-r--r--  1 root root   450695 Dec 15  2020 gurobi91.netstandard20.xml
-rw-r--r--  1 root root    86293 Dec 15  2020 gurobi.jar
-rw-r--r--  1 root root   159945 Dec 15  2020 gurobi-javadoc.jar
-rwxr-xr-x  1 root root      769 Dec 15  2020 gurobi.py
-rwxr-xr-x  1 root root 38305304 Dec 15  2020 libgurobi91_light.so
lrwxrwxrwx  1 root root       20 Dec 15  2020 libgurobi91.so -> ./libgurobi.so.9.1.1
lrwxrwxrwx  1 root root       20 Dec 15  2020 libgurobi_c++.a -> ./libgurobi_g++4.2.a
-rw-r--r--  1 root root   615816 Dec 15  2020 libgurobi_g++4.2.a
-rw-r--r--  1 root root   564336 Dec 15  2020 libgurobi_g++5.2.a
-rwxr-xr-x  1 root root   112203 Dec 15  2020 libGurobiJni91.so
-rwxr-xr-x  1 root root 42291983 Dec 15  2020 libgurobi.so.9.1.1
drwxr-xr-x  3 root root       55 Dec 15  2020 python2.7_utf16
drwxr-xr-x  3 root root       55 Dec 15  2020 python2.7_utf32
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.6_utf32
drwxr-xr-x 12 root root     5190 Dec 15  2020 python3.7
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.7_utf32
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.8_utf32
drwxr-xr-x  3 root root       55 Dec 15  2020 python3.9_utf32
-rwxr-xr-x  1 root root     1215 Dec 15  2020 rootcert.pem

here is the command I ran:

sirius/bin/sirius -i filtered_output_sirius.mgf formula -p orbitrap --ilp-solver gurobi

and this is the header of the standard output that I get:

Jan 08, 2024 9:24:24 AM org.apache.commons.beanutils.FluentPropertyBeanIntrospector introspect
INFO: Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Sirius Workspace Successfull initialized at: /nfs/users/nfs_f/fl4/.sirius-5.8
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: You run SIRIUS 5.8.5 on linux_amd64
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: You run SIRIUS in 'CLI' mode.
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Sirius was compiled with the following ILP solvers: GLPK-v1.7.0 (included), Gurobi-v9.1.1, CPLEX-v12.7.1, COIN-OR-v2.10.8.4
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Treebuilder priorities loaded from 'sirius.properties' are: [GUROBI]
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: CPU check done. 1 cores that handle 1 threads were found.
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.ChemistryBase.jobs.SiriusJobs getGlobalJobManager
INFO: Job manager successful initialized with 1 CPU thread(s) and 1 IO thread(s).
Jan 08, 2024 9:24:26 AM de.unijena.bioinf.rest.ProxyManager decorateWithPoolSettings
INFO: Starting http Client with MaxPerRoute=1 / maxTotal=5 (CPU-Threads=1).
Jan 08, 2024 9:24:27 AM de.unijena.bioinf.auth.AuthService getToken
WARNING: No login Found: Not Logged in, No valid refresh token Available
Jan 08, 2024 9:24:27 AM de.unijena.bioinf.ms.frontend.core.ApplicationCore <clinit>
INFO: Web API initialized.
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.ms.frontend.Run parseArgs
INFO: Running with following arguments: [-i, test_dataset_Claudio/filtered_output_sirius.mgf, formula, -p, orbitrap, --ilp-solver, gurobi]
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <0>[InputExpanderJJob-0] Running post processing
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <0>[InputExpanderJJob-0] Running cleanup
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logInfo$9
INFO: <1>[InputExpanderJJob-1] Expanding Input Files: 'test_dataset_Claudio/filtered_output_sirius.mgf'...
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logInfo$9
INFO: <1>[InputExpanderJJob-1] ...Input Files successfully expanded!
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <1>[InputExpanderJJob-1] Running post processing
Jan 08, 2024 9:24:28 AM de.unijena.bioinf.jjobs.JJob lambda$logDebug$13
FINE: <1>[InputExpanderJJob-1] Running cleanup
Jan 08, 2024 9:24:29 AM de.unijena.bioinf.ms.frontend.subtools.sirius.SiriusOptions setSolver
INFO: ILP solver changed to 'GUROBI' by command line.
...

and this is the error I get after a while (reiterated for all the compounds that are being assessed - Sirius does not crash, it just caries on failing like this):

Jan 08, 2024 2:43:35 PM de.unijena.bioinf.FragmentationTreeConstruction.computation.tree.TreeBuilderFactory getTreeBuilderFromClass
WARNING: Could not load GrbSolver! Sirius was compiled with the following ILP solvers: GLPK-v1.7.0 (included), Gurobi-v9.1.1, CPLEX-v12.7.1, COIN-OR-v2.10.8.4: Could not initialize class de.unijena.bioinf.FragmentationTreeConstruction.com
putation.tree.ilp.GrbSolver
Jan 08, 2024 2:43:35 PM de.unijena.bioinf.FragmentationTreeConstruction.computation.tree.TreeBuilderFactory getTreeBuilder
SEVERE: Your system does not ship with any instantiatable ILP solver. Please install either CLP,  Gurobi or CPLEX to use SIRIUS.
Jan 08, 2024 2:43:36 PM de.unijena.bioinf.jjobs.JJob lambda$logError$2

when terminating the job manually, I got this last error (which I suspect to be just due to user interuption, but mentioning it just in case):

SEVERE: <1265504>[RecalibrationJob-1265504] Failed!
java.lang.NullPointerException: Cannot invoke "de.unijena.bioinf.FragmentationTreeConstruction.computation.tree.TreeBuilder.computeTree()" because "finalBuilder" is null
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.recalibrate(FasterTreeComputationInstance.java:577)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance$RecalibrationJob.compute(FasterTreeComputationInstance.java:549)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance$RecalibrationJob.compute(FasterTreeComputationInstance.java:535)
        at de.unijena.bioinf.jjobs.BasicJJob.call(BasicJJob.java:117)
        at de.unijena.bioinf.jjobs.BasicMasterJJob$1.compute(BasicMasterJJob.java:101)
        at java.base/java.util.concurrent.RecursiveTask.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.awaitDone(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.join(Unknown Source)
        at de.unijena.bioinf.jjobs.BasicJJob.awaitResult(BasicJJob.java:211)
        at de.unijena.bioinf.jjobs.JJob.takeResult(JJob.java:249)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.estimateTreeSizeAndRecalibration(FasterTreeComputationInstance.java:307)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.compute(FasterTreeComputationInstance.java:173)
        at de.unijena.bioinf.FragmentationTreeConstruction.computation.FasterTreeComputationInstance.compute(FasterTreeComputationInstance.java:55)
        at de.unijena.bioinf.jjobs.BasicJJob.call(BasicJJob.java:117)
        at de.unijena.bioinf.jjobs.BasicMasterJJob$1.compute(BasicMasterJJob.java:101)
        at java.base/java.util.concurrent.RecursiveTask.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.awaitDone(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.join(Unknown Source)
        at de.unijena.bioinf.jjobs.BasicJJob.awaitResult(BasicJJob.java:211)
        at de.unijena.bioinf.sirius.Sirius$SiriusIdentificationJob.compute(Sirius.java:640)
        at de.unijena.bioinf.sirius.Sirius$SiriusIdentificationJob.compute(Sirius.java:623)
        at de.unijena.bioinf.jjobs.BasicJJob.call(BasicJJob.java:117)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

Can you help me with this please?

Many thanks,

Florent Lassalle Principal Bioinformatician PaM informatics Parasites and Microbes Programme Wellcome Sanger Institute

[mfleisch]

Hey we are currently checking this issue for SIRIUS 6 but we are likely not looking in to it for SIRIUS 5 anymore.