docker: improve Rust dependency caching in CI to avoid rebuilding

[gustavovalverde]

Motivation

When files are copied into the Dockerfile with the COPY command, the cache can get invalidated. We could improve this by using the --link option which creates more independent layers to avoid rebuilding when not needed

Specifications

Documentation: https://docs.docker.com/engine/reference/builder/#copy---link

[gustavovalverde]

Extra documentation:

• https://hackmd.io/@kobzol/S17NS71bh
• https://github.com/moby/moby/issues/39530
• https://github.com/rust-lang/cargo/issues/2644
• https://github.com/rust-lang/cargo/issues/6529
• https://github.com/Swatinem/rust-cache/issues/155

Partial solutions:

• https://ectobit.com/blog/rust-container-image-buildkit-buildx/
• https://github.com/paradigmxyz/reth/commit/a22c274394da913b327ad1bfdb2db008b272f0ab
• https://github.com/moby/buildkit/issues/1512#issuecomment-1617928647

[gustavovalverde]

This has become a cost issue, so we should prioritize it.

[gustavovalverde]

As caching is not working as expected, and using it takes the build more time as it's also pushing the cache to the registry, we decided to remove this here: https://github.com/ZcashFoundation/zebra/pull/8374/files#diff-5a5111df219975ea6b3b69567752b89fa0c670d5a67598674ad411da2ed96985L174-L187 in PR #8374 as it's generating costs for storing the artifacts without an actual value added.

This must be taken into consideration when working towards a solution.