cosima-cookbook icon indicating copy to clipboard operation
cosima-cookbook copied to clipboard

Deal with duplicated experiment name in querying

Open angus-g opened this issue 4 years ago • 4 comments

We try to limit the assumptions made in the database, so experiment names (like 01deg_jra55v13_ryf9091) aren't themselves unique — rather, the combination of experiment name and its root directory is. However, in functions like cc.querying.getvar, we only ask for the experiment name. This could lead to some unexpected behaviour, pulling files from two experiments with the same name, but a different root directory.

angus-g avatar Mar 23 '20 06:03 angus-g

Good point, sorry. Actually, that particular experiment name is unique it is just that (for a short time) there is a duplicate copy on both hh5 and ik11. So, I will delete the hh5 copy as soon as I can convince myself that the rsync went well.

AndyHoggANU avatar Mar 23 '20 08:03 AndyHoggANU

Actually, this was because I had two versions of the same folder listed on hh5, one with /g/data/hh5/ and one with /g/data3/hh5/. This should be easy to avoid...

rmholmes avatar Mar 23 '20 21:03 rmholmes

Should be easy to avoid, but I don't think it should break so horribly in these cases!

angus-g avatar Mar 23 '20 21:03 angus-g

The idea of a unique experiment id could address this I think. In the case where there is match with multiple IDs the operation could error and the user warned that manual disambiguation is required. The same thing could be done if there are multiple root paths I guess.

In any case, definitely a related issue

https://github.com/payu-org/payu/issues/191

aidanheerdegen avatar Sep 01 '20 05:09 aidanheerdegen