turbo icon indicating copy to clipboard operation
turbo copied to clipboard

[turborepo] with-docker example's Dockerfiles violating best practices

Open NexZhu opened this issue 3 years ago • 2 comments

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

I think the current with-docker example's Dockerfiles is not good for utilizing Docker build cache.

# Set working directory
COPY . .
RUN turbo prune --scope=web --docker

This will copy all files in the monorepo project, which takes more time than neccessary, also if any other package change, even if that package is not a dependency of the current package, it will invalidate the build cache, causing all statements after COPY . . to run again.

Describe the solution you'd like

I think RUN turbo prune --scope=web --docker should run outside of the Docker container, and only COPY the .out directory which will invalidate Docker build cache only if any related file actually changes.

Describe alternatives you've considered

.

NexZhu avatar Nov 16 '22 08:11 NexZhu

@NexZhu Care to elaborate why you closed this issue? I think I'm running into exactly the same problem. Docker builds are taking ages with TurboRepo because basically the whole Docker caching mechanism is circumvented.

soulchild avatar Nov 18 '22 12:11 soulchild

@soulchild Because I hadn't look at the Dockerfile carefully before I opened the issue, after taking a look again, I think the builder stage itself runs very fast, and if the content of the out directory doesn't change (if some unrelated packages change, it won't effect the out directory content), Docker build cache of the COPY --from=builder /app/out/xxx commands in the install stage won't be invalidated, so overall it shouldn't slow down the build.

But if you found it taking ages, maybe I was wrong this time, and there are actually optimizations needed to be done to the Dockerfile, but I don't know where. I've reopend the issue so people can look into it more closely.

NexZhu avatar Nov 18 '22 15:11 NexZhu

@NexZhu You're right: For whatever reasons, repeated build runs now use the Docker cache correctly (from the installer stage onwards). Maybe I had something else tripping up the build process…

soulchild avatar Nov 21 '22 07:11 soulchild