Add new tools to rptools suite and update old tools

[kenza12]

Add new tools to rptools suite:

• rpscore
• rpthermo
• rpcompletion

Update rptools suite to the latest version:

• rpextractsink
• rpranker
• rpreport
• rpviz
• rpfba

FOR CONTRIBUTOR:

• [ ] - I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
• [ ] - License permits unrestricted use (educational + commercial)
• [ ] - This PR adds a new tool or tool collection
• [ ] - This PR updates an existing tool or tool collection
• [ ] - This PR does something else (explain below)

[kenza12]

It seems that planemo test is comparing old outputs.

I got this error in rpranker:

Output sorted_pathways:  different than expected, difference (using diff):
( /tmp/tmpt80blccplycopene_ranked_pathways.csv v. /tmp/tmpbjgo3f84lycopene_ranked_pathways.csv )
--- local_file
+++ history_data
@@ -1,10 +1,4 @@
 #Name,Score
-001_0001,0.9850856959819794
-001_0006,0.9850397169589996
-001_0011,0.9850397169589996
-002_0001,0.9892438888549804
-002_0011,0.9851486444473266
-002_0021,0.9851486444473266
-003_0001,0.975385755300522
-003_0131,0.980250358581543
-003_0261,0.9863117098808288
+rp_003_0001.xml,0.975385755300522
+rp_003_0131.xml,0.980250358581543
+rp_003_0261.xml,0.9863117098808288

And this is the command line:

ln -s /tmp/tmp8j7lk5kg/files/000/dataset_10.dat rp_003_0261.xml && ln -s /tmp/tmp8j7lk5kg/files/000/dataset_11.dat rp_003_0131.xml && ln -s /tmp/tmp8j7lk5kg/files/000/dataset_12.dat rp_003_0001.xml && python -m rptools.rprank --pathways rp_003_0261.xml rp_003_0131.xml rp_003_0001.xml > '/tmp/tmp8j7lk5kg/job_working_directory/000/13/outputs/galaxy_dataset_24519a89-a784-4bc9-b937-b52dfa513ba5.dat'

Planemo try to compare with old pathways as 002_0001 or 001_0001 .. and they are not existing anymore. As you can see in the command line, the needed pathways are: rp_003_0001 , rp_003_0131 and rp_003_0261 .

Same issue with rpthermo where he try to compare with a non existing output file test. I deleted it in the last commit.

[kenza12]

It seems that planemo test is comparing old outputs.

I got this error in rpranker:

Output sorted_pathways:  different than expected, difference (using diff):
( /tmp/tmpt80blccplycopene_ranked_pathways.csv v. /tmp/tmpbjgo3f84lycopene_ranked_pathways.csv )
--- local_file
+++ history_data
@@ -1,10 +1,4 @@
 #Name,Score
-001_0001,0.9850856959819794
-001_0006,0.9850397169589996
-001_0011,0.9850397169589996
-002_0001,0.9892438888549804
-002_0011,0.9851486444473266
-002_0021,0.9851486444473266
-003_0001,0.975385755300522
-003_0131,0.980250358581543
-003_0261,0.9863117098808288
+rp_003_0001.xml,0.975385755300522
+rp_003_0131.xml,0.980250358581543
+rp_003_0261.xml,0.9863117098808288

And this is the command line:

ln -s /tmp/tmp8j7lk5kg/files/000/dataset_10.dat rp_003_0261.xml && ln -s /tmp/tmp8j7lk5kg/files/000/dataset_11.dat rp_003_0131.xml && ln -s /tmp/tmp8j7lk5kg/files/000/dataset_12.dat rp_003_0001.xml && python -m rptools.rprank --pathways rp_003_0261.xml rp_003_0131.xml rp_003_0001.xml > '/tmp/tmp8j7lk5kg/job_working_directory/000/13/outputs/galaxy_dataset_24519a89-a784-4bc9-b937-b52dfa513ba5.dat'

Planemo try to compare with old pathways as 002_0001 or 001_0001 .. and they are not existing anymore. As you can see in the command line, the needed pathways are: rp_003_0001 , rp_003_0131 and rp_003_0261 .

Same issue with rpthermo where he try to compare with a non existing output file test. I deleted it in the last commit.

This is weird, I still get the same error with rpranker, and I got a new error with rpThermo (which passed before):

Timed out after 900.25 seconds waiting on tool test run.

[yorgomoubayed]

@bgruening is it possible to follow up on this please, any recommendations or suggestions? Thank you.

[bgruening]

I restarted the tests and you see that all are failing with:

/usr/local/lib/python3.9/site-packages/cobra/io/mat.py:13: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  from numpy import object as np_object
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/site-packages/rptools/rpviz/__main__.py", line 20, in <module>
    from rptools.rpviz.utils import annotate_cofactors, annotate_chemical_svg, get_autonomous_html, parse_all_pathways
  File "/usr/local/lib/python3.9/site-packages/rptools/rpviz/utils.py", line 18, in <module>
    from rptools.rplibs import rpSBML, rpPathway
  File "/usr/local/lib/python3.9/site-packages/rptools/rplibs/__init__.py", line 7, in <module>
    from rptools.rplibs.rpPathway import rpPathway
  File "/usr/local/lib/python3.9/site-packages/rptools/rplibs/rpPathway.py", line 46, in <module>
    from .rpSBML import rpSBML
  File "/usr/local/lib/python3.9/site-packages/rptools/rplibs/rpSBML.py", line 40, in <module>
    import cobra
  File "/usr/local/lib/python3.9/site-packages/cobra/__init__.py", line 17, in <module>
    from cobra import io
  File "/usr/local/lib/python3.9/site-packages/cobra/io/__init__.py", line 6, in <module>
    from cobra.io.mat import load_matlab_model, save_matlab_model
  File "/usr/local/lib/python3.9/site-packages/cobra/io/mat.py", line 13, in <module>
    from numpy import object as np_object
ImportError: cannot import name 'object' from 'numpy' (/usr/local/lib/python3.9/site-packages/numpy/__init__.py)

You can download the error summary from the GH pages. I guess numpy needs to be pinned to a previous version or the code needs to be adopted?

[guillaume-gricourt]

Hi, the wrapper rpthermo is failing after timeout without logging anything in the artifacts. Locally, this error is not thrown by this command line:

planemo test \
--biocontainers \
--no_dependency_resolution \
--no_conda_auto_init \
--galaxy_source https://github.com/galaxyproject/galaxy \
--galaxy_branch release_22.05 \
--galaxy_python_version 3.7 \
--test_timeout 900 \
rpthermo.xml

How can I solve this ?

[bgruening]

How long is it running locally? Maybe its really timing out here on CI.

[guillaume-gricourt]

Locally, the test takes roughly 5 minutes and some logs are created. Here, it seems there is no logs produced and running time out. So, I don't know where to search to solve this problem. I'll take any ideas to try something. Many thanks

[guillaume-gricourt]

conda seems to be very slow to install rptools with the classic solver and test are running time out. May be related to #5078

[bgruening]

You can now see the test failures at https://github.com/galaxyproject/tools-iuc/actions/runs/7916833069?pr=5005 at the bottom.