kit icon indicating copy to clipboard operation
kit copied to clipboard

"Memoization" / work avoidance / don't run task if sources haven't changed

Open agilgur5 opened this issue 1 year ago • 4 comments

Mentioned in https://github.com/kitproj/kit/issues/40#issuecomment-1755913391 but didn't get an answer. I think this behavior is not currently supported.

Task supports specifying sources and generates and will perform a checksum or timestamp (checksum is the default, timestamp matches Make) check on those when specified. If the files listed in generates exist and the sources haven't changed, the task will not be run.

Make supports a similar feature by default that uses timestamps.

Would be good feature parity for Kit to support this, especially as it can result in using a lot less compute and time

agilgur5 avatar Oct 25 '23 18:10 agilgur5

This seems like a really good idea. File hashes will be expensive to compute if there are many files, so we could use the modtime. I think this is what Make does.

alexec avatar Jul 15 '24 18:07 alexec

Note, this is only needed when starting up. The way that Make does this is that it compares the built file timestamp with the source file. If the built file is newer, then it is up-to-date, older then rebuild is needed.

alexec avatar Jul 15 '24 18:07 alexec

What about this syntax

# we watch these files
watch: [foo]
# when the task runs to completion and exit code 0, touch this file
touch: bar

alexec avatar Jul 15 '24 18:07 alexec

so we could use the modtime. I think this is what Make does.

Yes that's what I wrote above. Although modtime is imperfect as it will trigger when you save a file and there were no changes.

Note, this is only needed when starting up.

Does watch already check for this condition?

What about this syntax

I would think that if there's a watch, there's no need for another field, as those are the ones you'd check the timestamp or checksum against.

For tasks without outputs though, that might be useful

agilgur5 avatar Jul 15 '24 23:07 agilgur5