spyglass
spyglass copied to clipboard
Populate should leave evidence of success state
Problem: It is possible to run common_behav.StateScript.populate
for a given key and have no resulting insert. This leaves previously attempted keys in the key_source
, which will be re-attempted if re-run. The current structure doesn't distinguish between 'attempted' and 'not attempted'.
Proposed: By modifying the secondary key to be nullable, we can retain a record of which keys have been attempted. Optionally, another secondary key could be added to be explicit (e.g., has_assoc_file: bool
). I have seen this pattern on other tables. I propose migrating all populates
to retain records of attempted keys with null entries
Considerations: In this case the cost of rerunning is small, a couple conditionals, but this habit opens the door to unnecessary processing time elsewhere. Stylistically, lab members tend to populate a specific key as needed, which will not proc this issue, but datajoint documentation often uses bare populates, so use habits may change as more folks try out spyglass