tools icon indicating copy to clipboard operation
tools copied to clipboard

Replace check_max function with resourceLimits directive

Open jfy133 opened this issue 1 year ago • 3 comments

Closes #2923

PR checklist

  • [ ] This comment contains a description of changes (with reason)
  • [ ] CHANGELOG.md is updated
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
  • [ ] Documentation in docs is updated

jfy133 avatar Jun 25 '24 12:06 jfy133

This PR is against the master branch :x:

  • Do not close this PR
  • Click Edit and change the base to dev
  • This CI test will remain failed until you push a new commit

Hi @jfy133,

It looks like this pull-request is has been made against the jfy133/nf-core-tools master branch. The master branch on nf-core repositories should always contain code from the latest release. Because of this, PRs to master are only allowed if they come from the jfy133/nf-core-tools dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page. Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

github-actions[bot] avatar Jun 25 '24 12:06 github-actions[bot]

I tested locally by running the nf-core/test-pipeline example in the CI.

By adjusting --max_memory to 16, the FASTQC process that would normally get 32GB (label_medium) was indeed capped at 16 (normally it would be allocated 32)

image

jfy133 avatar Jun 25 '24 13:06 jfy133

Let's just remove params.max_* entirely 🔥

Just don't remove me please

maxulysse avatar Jun 25 '24 13:06 maxulysse

OK the tasks from the slack thread here

  • [x] Remove all references to --max_* parameters
  • [x] Add lint tests (so people remove the parameters)
  • [x] Set new defaults in tests configs
  • [ ] Update the docs on the website
    • [ ] Add warning about legacy useage
    • [ ] New docs to describe how to use new resourceDirectives
  • [ ] mass update all configs on nf-core/configs
    • [ ] Keep old --max_* parameters (buit leave comment)
    • [ ] Copy vbalues to new resourcesLimits (with

jfy133 avatar Sep 04 '24 09:09 jfy133

@nf-core-bot fix linting

jfy133 avatar Sep 04 '24 09:09 jfy133

This is ready to merge, ok if we merge it @jfy133 ? We have to change the branch protection for the two pending tests, but I want to do it right before merging, otherwise it will block other PRs.

mirpedrol avatar Sep 19 '24 12:09 mirpedrol

Yes I think we can do but we Just have to be careful that people don't try to use dev to generate pipelines because configs won't work, I'll try and work out a mass configs PR ASAP.

Docs can come later I guess

jfy133 avatar Sep 19 '24 14:09 jfy133

ok I will merge it now! It won't affect old pipelines so it should be good 👍

mirpedrol avatar Sep 20 '24 07:09 mirpedrol