oncoanalyser icon indicating copy to clipboard operation
oncoanalyser copied to clipboard

Shrink reference data to support execution on GHA runners

Open FriederikeHanssen opened this issue 1 year ago • 1 comments

Edited issue description (SW)

  • currently reference data required to run the mini simulated test data remains at 100+ GB
  • this far exceeds disk capacity of GHA runners and because of this we cannot currently run non-stub tests for CI
  • shrinking reference data will likely take several weeks and this work is to be deferred until after the 1.0.0 release

For initial discussions, please see https://github.com/nf-core/oncoanalyser/pull/29#discussion_r1683962125


Original issue description (FH) [click to show]
          I've been unable to run an end-to-end test for the CI since the required reference is too large for the GHA runners. And you're right that the 30.GB request is also too high for the default runners. Instead, this is something I run locally for testing.

The current test executed by the GHA workflow is only a stub test. This is something I planned to improve after the initial release - I suspect it will take a significant amount of time to reduce reference data size so that end-to-end tests work on the restrictive default runners

Originally posted by @scwatts in https://github.com/nf-core/oncoanalyser/pull/29#discussion_r1683962125

FriederikeHanssen avatar Jul 30 '24 18:07 FriederikeHanssen

Thanks for creating opening this @FriederikeHanssen! I've made some edits above to give some more context for when we return to the issue

scwatts avatar Jul 31 '24 07:07 scwatts

This unfortunately requires changes in the upstream WiGiTS tools to be feasible

scwatts avatar Sep 04 '25 01:09 scwatts