FluidFramework icon indicating copy to clipboard operation
FluidFramework copied to clipboard

improvement(fluid-build): Use hash-based caching for type test task

Open tylerbutler opened this issue 1 year ago • 1 comments

The type-test generation tasks are particularly susceptible to cache invalidation in fluid-build, because it uses a filestats-based cache and they include package.json files in the cache. Since package.json is often touched when changing branches, this can make operations like checking out a branch, then checking out the previous branch again having made no changes cause rebuilds. In the current implementation, such actions invalidate the cache.

This change updates the type-test task to use a hash-based cache.

This mechanism is ~20% slower than the stats-based approach, but tasks with few files/smaller files are less affected. The type-test task only has two inputs and few outputs.

tylerbutler avatar Jul 15 '24 19:07 tylerbutler

There is an invalidation issue with this task and .done tracking. That should be understood before further consideration of this change. In the listed scenario, the assumption appears to be that there is no build while away from the original branch. That feels a little suspect. I know that is does happen, but the trade-off needs to be considered. If that happens only 5% of situations across the team, is a 20% hit (to dep check) worth it 100% of the time? Additionally, while this task does not have the true and correct set up inputs identified, running more often is better.

jason-ha avatar Jul 15 '24 19:07 jason-ha

This PR has been automatically marked as stale because it has had no activity for 60 days. It will be closed if no further activity occurs within 8 days of this comment. Thank you for your contributions to Fluid Framework!