hardhat

hardhat copied to clipboard

To reduce the number of CI fails due to a solc compiler download failure we should add OS specific cache for the solc cache folder.

We shouldn't need cache invalidation as the solc compiler is always the same for each version. Though we should consider if there is a nice way of including these caches in lots of workflows.

https://github.com/NomicFoundation/hardhat/blob/main/packages/hardhat-core/src/internal/solidity/compiler/downloader.ts

https://github.com/NomicFoundation/hardhat/blob/f6c41e4f14bc7e246fb5da6110c25be8824e5d15/packages/hardhat-core/src/internal/solidity/compilation-job.ts#L6

https://github.com/NomicFoundation/hardhat/commit/ab34a7ba0fd979c462f37194aec97cea987aa8e3

https://github.com/NomicFoundation/hardhat/commit/220f2c88e13e392f7cfcf3ed944aed89dfa898ad

We need multiple cache dirs, one for each OS.

Example(s):

cache-v2 -> cache_macosx-amd64 ... ->cache_wasm

or convention on ENUM pattern:

cache-v2 -> cache-MACOS ... -> cache-WASM

or lastly we have cache folders within named for OS.

We then update references to the cache-v2 folder to a subdir or new naming convention, with subdir we use path.join

It already sort of does this:

https://www.npmjs.com/package/env-paths

running npx hardhat compile caches at ~/.cache/hardhat-nodejs/compilers-v2 like:

linux-amd64

But that's redundant because of path-env package, The concept isn't because of compiler version. My situation might be different and at first it looked like I needed to call getCompilersDir in the /hardhat/internal/util dir and I do have the solc version specified in hardhat.config.js under solidity.compilers[].version.

Conclusion: The CI fails are due to the container not allowing access to the path-env defined location. The correct method to deal with this with security in mind is to avoid copying versions to different OS's and to instead allow for the cache dir to place in a custom directory, local directory (has issues), update path-env to deal with container constraints, or re-arch with the added benefit of avoiding potential directory traversal if we remove path-env to run locally. I don't think this is feasible.

We should probably just have documentation to set path-env if that was the issue.

I understand that the VM with the container might be what you're after, so I'll look through and see if the compiler object in the config can additionally support an OS type like:

[ { version: "0.8.24", os: "linux-amd64" } ]

lastly...

We'll need to select the right version every time, but I think the current method does os.~whatever~ to get that linux-amd64 POSIXish string.

https://github.com/NomicFoundation/hardhat/blob/c7890f0b75a685fc95c37693ecc76af3fd1c46fe/packages/hardhat-core/src/internal/solidity/compiler/downloader.ts#L112

This issue is targeted at the caching for Hardhat's own CI tests, rather than projects that use Hardhat. Because we don't have caching setup for compilers in Hardhat's github actions, we can get nondeterministic fails if the solc compiler download fails. This issue is about making our CI on Hardhat itself more robust.

We are going to hold on this - our CI has been reworked and several issues fixed. We will let that bed in and reassess.