fetchngs icon indicating copy to clipboard operation
fetchngs copied to clipboard

Check if running offline

Open drpatelh opened this issue 3 years ago • 1 comments

This pipeline won't work in offline situations because it needs to download metadata and fastq files from the web. We should probably add a check/warning/fail to make sure that there is an internet connection when running. This would have to factor in situations where login nodes could have an internet connection but the nodes on the cluster may be offline. We should be able to add try/catch logic to the bin/sra_ids_to_runinfo.py script to detect this or maybe we just write a separate module for this that can be re-used if required as this will only ever by run once at the beginning of the workflow with almost zero resources.

drpatelh avatar Jun 29 '21 09:06 drpatelh

Hi Harshil,

This issue was originally opened due to my post on nf-core slack. To diagnose the issue you created an example script that would download data via sbatch as a quick check of whether nodes on my cluster had internet access. Indeed we found with this script that they had internet access. Despite my compute nodes having internet access, the issue persists (even with recent attempts on pipeline v1.2). Leads me to believe it may be something else such as something unique about the nextflow/singularity environments. Any additional things to try would be helpful, but mostly wanted to comment and bring this up for your consideration when addressing this issue in the future.

Thank you! Ryan

ryanmboileau avatar Aug 18 '21 22:08 ryanmboileau