Next generation mirror-intel

[skyzh]

mirror-intel is the state-of-art tool for on-demand mirroring and serving contents from s3 storage. But currently, we found a lot of issues that blocked us from updating mirror-intel frequently.

• Every change in cache rule requires a re-compile, which takes 10 minutes.
• Rocket.rs is too high-level that made custom URL-parsing a headache.
• Reload configuration requires a restart.
• Listing page is ugly.
• No TTL for the cache.
• ...

So here I propose the next-generation mirror-intel, which may possibly solve the issues above.

• Use a low-level HTTP programming framework like https://github.com/seanmonstar/warp, where we could skip the complex routing and parsing scheme provided by high-level frameworks and get URL parameters directly.
• Use a local database to manage all files, thus allowing removing files after a given TTL.
• Use a rule-based configuration file and support hot-reload.

# path regex, policy, upstream
/guix/(.*)\.nar, proxy-cache, guix.org
/guix/(.*), redirect-to, guix.org
/homebrew-bottles/(.*), cache, xxxxx

• Support symbolic links in cooperation with mirror-clone