bioconda-recipes icon indicating copy to clipboard operation
bioconda-recipes copied to clipboard

Update comet-ms version in openms-thirdparty

Open jonasscheid opened this issue 3 months ago • 20 comments

Follow-up of #46567


Please read the guidelines for Bioconda recipes before opening a pull request (PR).

General instructions

  • If this PR adds or updates a recipe, use "Add" or "Update" appropriately as the first word in its title.
  • New recipes not directly relevant to the biological sciences need to be submitted to the conda-forge channel instead of Bioconda.
  • PRs require reviews prior to being merged. Once your PR is passing tests and ready to be merged, please issue the @BiocondaBot please add label command.
  • Please post questions on Gitter or ping @bioconda/core in a comment.

Instructions for avoiding API, ABI, and CLI breakage issues

Conda is able to record and lock (a.k.a. pin) dependency versions used at build time of other recipes. This way, one can avoid that expectations of a downstream recipe with regards to API, ABI, or CLI are violated by later changes in the recipe. If not already present in the meta.yaml, make sure to specify run_exports (see here for the rationale and comprehensive explanation). Add a run_exports section like this:

build:
  run_exports:
    - ...

with ... being one of:

Case run_exports statement
semantic versioning {{ pin_subpackage("myrecipe", max_pin="x") }}
semantic versioning (0.x.x) {{ pin_subpackage("myrecipe", max_pin="x.x") }}
known breakage in minor versions {{ pin_subpackage("myrecipe", max_pin="x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
known breakage in patch versions {{ pin_subpackage("myrecipe", max_pin="x.x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
calendar versioning {{ pin_subpackage("myrecipe", max_pin=None) }}

while replacing "myrecipe" with either name if a name|lower variable is defined in your recipe or with the lowercase name of the package in quotes.

Bot commands for PR management

Please use the following BiocondaBot commands:

Everyone has access to the following BiocondaBot commands, which can be given in a comment:

@BiocondaBot please update Merge the master branch into a PR.
@BiocondaBot please add label Add the please review & merge label.
@BiocondaBot please fetch artifacts Post links to CI-built packages/containers.
You can use this to test packages locally.

Note that the @BiocondaBot please merge command is now depreciated. Please just squash and merge instead.

Also, the bot watches for comments from non-members that include @bioconda/<team> and will automatically re-post them to notify the addressed <team>.

jonasscheid avatar Mar 22 '24 14:03 jonasscheid

@jpfeuffer @timosachsenberg Have you ever seen this before? Do we hit a space limit now? That's new. Build should not be bigger than the previous one 🤔

.15:17:08 BIOCONDA INFO (ERR) [Mar 22 15:17:08] SERR [Mar 22 15:17:08] ERRO Task processing failed: API error (500): {"message":"write /usr/local/share/sirius-ms-5.8.2-0/app/sirius_rest_service-5.8.2-boot.jar: no space left on device"}

jonasscheid avatar Mar 22 '24 17:03 jonasscheid

Yes. I pray for the day they throw out mulled container testing.

jpfeuffer avatar Mar 22 '24 17:03 jpfeuffer

How do we move forward?

jonasscheid avatar Mar 23 '24 12:03 jonasscheid

I honestly don't know. We never had this problem and our own package size rather decreased than increased. Dependencies must have increased in size or the basic container itself uses more space now.

jpfeuffer avatar Mar 23 '24 15:03 jpfeuffer

@bioconda/core Is there any workaround here?

jonasscheid avatar Mar 23 '24 18:03 jonasscheid

Reposting for @jonasscheid to enable pings (courtesy of the BiocondaBot):

@bioconda/core Is there any workaround here?

BiocondaBot avatar Mar 23 '24 18:03 BiocondaBot

@bgruening any idea?

timosachsenberg avatar Mar 26 '24 22:03 timosachsenberg

Since bioconda has a hard limit on package size due to its container tests, the only two solutions that I see:

  • move to conda-forge
  • reduce the size (e.g. by removing Sirius)

jpfeuffer avatar Apr 06 '24 11:04 jpfeuffer

I would say we can remove Sirius.

timosachsenberg avatar Apr 07 '24 15:04 timosachsenberg

It's quite suboptimal since the same minor version will have different thirdparty tools but we have no choice if bioconda can't even handle around 1GB of packages that don't even have to be tested since openms-thirdparty is just an agglomeration of packages basically. And from my experience, I don't think anything will change soon.

jpfeuffer avatar Apr 07 '24 16:04 jpfeuffer

So we go for removing Sirius?

jonasscheid avatar Apr 19 '24 08:04 jonasscheid

conda-forge and also bioconda splits big packages into subpackages, different recipes if its needed.

bgruening avatar Apr 19 '24 08:04 bgruening

The openms-thirdparty recipe has 0 bytes though. It just wraps versions of dependencies that should be tested to work together.

jpfeuffer avatar Apr 19 '24 16:04 jpfeuffer

After removing Sirius it looks like there is some sort of hick-up when building 🤔

jonasscheid avatar May 01 '24 08:05 jonasscheid

Try to push some comment somewhere to see if the problem goes away.

jpfeuffer avatar May 01 '24 09:05 jpfeuffer

Seems a persistent error. @jpfeuffer @timosachsenberg Do you think we can outsource this bundle to multi-package-containers ?

jonasscheid avatar May 06 '24 19:05 jonasscheid

Sure, but a) the current error seems to be during the build of the regular openms package (probably stalling because out of RAM or something) b) multi-package container will not give you a conda package if you need that, too

jpfeuffer avatar May 06 '24 19:05 jpfeuffer

Work is still ongoing to remove mamba, support the new smaller package formats etc ... so I try restarting this build here. What strikes me is that this does not to be a space issue anymore? Is this hitting now some other limitation (memory, runtime)?

bgruening avatar May 06 '24 19:05 bgruening

Not sure what's going on with the CI since the logs have timed out @bgruening can you try to trigger the tests again?

poshul avatar May 13 '24 14:05 poshul

##[error]No hosted parallelism has been purchased or granted. To request a free parallelism grant, please fill out the following form https://aka.ms/azpipelines-parallelism-request

@bgruening I guess this is a bioconda issue? when will this be resolved

timosachsenberg avatar May 13 '24 15:05 timosachsenberg