funflow
funflow copied to clipboard
tweag
Working out what has caused steps to rerun should be easier
I have quite a simple flow. The first step is impure as it scrapes the web to find the work that it needs to do. The output from the first step is a directory of files, each file is a specification for some work that needs to be done. In theory, only specifications which have been changed will be reprocessed but in practice I see quite a lot of recompilation. Printing out some message about what caused a step to be rerun would be very useful.
I suppose I just want a summary which states which steps were cached and which were rerun. Knowing that a step was cached is quite a good sanity check when you think it should be and likewise knowing which are rerun can give pointers to try to fix the recompilation in future.
