osemosys_global icon indicating copy to clipboard operation
osemosys_global copied to clipboard

[BUG] Deleting PLEXOS input datafiles breaks snakemake

Open trevorb1 opened this issue 3 years ago • 0 comments

Conda environment check

  • [X] I have tried using the osemosys-global conda environment.

Current Behavior

If you delete an input file, such as PLEXOS_World_MESSAGEix_GLOBIOM_Softlink.xlsx, and run the snakemake workflow an error will be thrown. The scripts have try except statements to check for the file locally first and only download if necessary. This means the scripts run fine independently, but if the file needs to be downloaded snakemake won't see that and break.

Expected Behavior

If you delete the datafile in resources/data and run the snakemake workflow, the appropriate files will be downloaded and the workflow will run without issue.

Steps To Reproduce

  1. Delete the file resources/data/PLEXOS_World_MESSAGEix_GLOBIOM_Softlink.xlsx
  2. Run the workflow with snakemake -c

Operating System

Linux

Log output

(osemosys-global) trevorb1@DESKTOP-HCG4NHT:~/repositories/osemosys_global$ snakemake -c
Building DAG of jobs...
MissingInputException in line 175 of /home/trevorb1/repositories/osemosys_global/workflow/rules/preprocess.smk:
Missing input files for rule max_capacity:
resources/data/PLEXOS_World_MESSAGEix_GLOBIOM_Softlink.xlsx

Anything else?

Maybe we should create a script that does all the file downloading right at the start of the workflow?

trevorb1 avatar Mar 16 '22 15:03 trevorb1