turbo icon indicating copy to clipboard operation
turbo copied to clipboard

Not caching with identical inputs

Open iamnafets opened this issue 1 year ago • 5 comments

Verify canary release

  • [X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

None.

What package manager are you using / does the bug impact?

Yarn v2/v3 (node_modules linker only)

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

1.11.3-canary.0

Describe the Bug

Re-running turbo multiple times produces a cache miss consistently in the same package (1 of 20). The outputs using --summarize show identical hash and hashOfExternalDependencies but both have status:MISS.

image image

Expected Behavior

This should be cached as the inputs are unchanged, the dependencies are unchanged, and I've run it multiple times.

To Reproduce

Unclear. I could use some guidance on places to look here.

Additional context

No response

iamnafets avatar Dec 20 '23 21:12 iamnafets

I'll try to create a repro repository. Any hints in the meantime would be helpful.

iamnafets avatar Dec 20 '23 21:12 iamnafets

Hey @iamnafets, thanks for the report - if you can create a reproduction starting from npx create-turbo@latest, and then pushing that up to a public repo with any additional instructions to reproduce that would be very helpful. Thanks!

tknickman avatar Dec 20 '23 21:12 tknickman

--summarize output could be wrong. The timeSaved: 0 reminds me of a bug where if the daemon didn't detect any changes to outputs, it would skip restoring the cache. In that case, we wouldn't know how much time you saved on this cache hit (because we never checked the cache!). I had fixed this before in the Go codepath. That timeSaved: 0, if it is indeed the same code path, it's possible that this cache:miss property is getting populated incorrectly.

Do you see FULL TURBO in the output (not in --summarize) when you run those two builds?

mehulkar avatar Dec 22 '23 18:12 mehulkar

No FULL TURBO because it's missing the cache for this particular package. I managed to bisect this back to a symlink, it seems like if you have a symlink in the inputs, despite them having the same content it blows the cache. I unfortunately don't have time to set up a repro repository for this since I've been able to unblock myself but effectively I had 2 workspaces:

A/foo B/foo -> ../A/foo

B's cache was always a miss.

iamnafets avatar Dec 22 '23 20:12 iamnafets

@iamnafets I tried reproducing with a symlink with 1.11.4-canary.1, and was not able to reproduce: https://github.com/mehulkar/turbo-issue-6833.

If this minimal reproduction matches your setup, can you try the new canary? We have seen some recent issues with "long" symlinks names that may be the cause of your problem as well, so if you could give more realistic paths for your case, that would help. Some of these issues have been fixed in the newest canary however, so please try that and send over any other information you have!

mehulkar avatar Jan 24 '24 14:01 mehulkar