fetchngs
fetchngs copied to clipboard
Check if running offline
This pipeline won't work in offline situations because it needs to download metadata and fastq files from the web. We should probably add a check/warning/fail to make sure that there is an internet connection when running. This would have to factor in situations where login nodes could have an internet connection but the nodes on the cluster may be offline. We should be able to add try/catch
logic to the bin/sra_ids_to_runinfo.py
script to detect this or maybe we just write a separate module for this that can be re-used if required as this will only ever by run once at the beginning of the workflow with almost zero resources.
Hi Harshil,
This issue was originally opened due to my post on nf-core slack. To diagnose the issue you created an example script that would download data via sbatch as a quick check of whether nodes on my cluster had internet access. Indeed we found with this script that they had internet access. Despite my compute nodes having internet access, the issue persists (even with recent attempts on pipeline v1.2). Leads me to believe it may be something else such as something unique about the nextflow/singularity environments. Any additional things to try would be helpful, but mostly wanted to comment and bring this up for your consideration when addressing this issue in the future.
Thank you! Ryan