comses.net icon indicating copy to clipboard operation
comses.net copied to clipboard

improve data / code organization in the archive

Open alee opened this issue 6 years ago • 0 comments

We should improve how we handle input data dependencies and code.

A continual source of confusion for model authors requesting peer review is how to refer to the data files the submit (in the data slot).

Right now code is placed in

./code/model.py

and data is placed in

./data/quux.asc

In order for model.py to access quux.asc it needs to use the relative path ../data/quux.asc. Frequently source code will use ./data/quux.asc or something along those lines though.

We could try adding a symlink in the ./code/ directory to the data directory if the ./code/data/ directory doesn't already exist.

E.g.,

.
├── code
│   ├── data -> ../data
│   └── model.py
└── data
    └── quux.asc

This would let model authors continue to refer to their data via the relative path ./data/quux.asc. Thoughts @comses/editors?

Then the technical part about putting a symlink in the zip archive we build is a bit hairy but seems possible: https://stackoverflow.com/questions/35782941/archiving-symlinks-with-python-zipfile

alee avatar May 29 '19 23:05 alee