cache-http icon indicating copy to clipboard operation
cache-http copied to clipboard

wish: automatic cache pruning

Open markstos opened this issue 3 years ago • 7 comments

To make this service more "just works", it could automatically prune the cache. Here's how that could work:

  • Right after a new upload completes and a success response has been returned to the client, A check could be made on the cache size or the number of items in the cache. For example, "If the number of the cached items exceeds 10, prune the cache". or "Prune items older than 10 days". This could possibly be it's own route that action-cache-http issues a "fire and forget" request to. (It issues the request but doesn't wait for the response).
  • Configuration could happen via the action-cache-http project. The pruning configuration would be passed to the /prune route if a custom pruning rule is needed.

markstos avatar Nov 10 '21 15:11 markstos

Normally, we prune it by cron tab. Diskspace is cheap, memory is expensive. Plus for around 20 projects, we stay in the range of 20GB ~ 50 GB. A crontab just takes care of those things.

Having such feature, would be a neat feature of course. Something like keep_days: 7.

kevincobain2000 avatar Nov 11 '21 04:11 kevincobain2000

Memory is expensive but if the cache service runs as a "service container", it's only running when it's needed.

markstos avatar Nov 11 '21 15:11 markstos

Or it can be done as

cache_http_api: "http://127.0.0.1:3000?keep_days=7"

This can be handled at the time of request

kevincobain2000 avatar Nov 12 '21 04:11 kevincobain2000

https://github.com/actions/cache/issues/505#issuecomment-1123435993

kevincobain2000 avatar May 13 '22 11:05 kevincobain2000

@kevincobain2000 Do the recent changes to action/cache obsolete this project? I asked on the linked ticket, but I presume you linked that ticket here because those changes impact this project.

markstos avatar May 13 '22 16:05 markstos

Hi @markstos, yes if GHES can use action/cache upon 3.5 GHE release, then this project will become redundant. Although, of course the action/cache, will need to lift off the GHES check, which seems like they have already done it https://github.com/actions/cache/blob/136d96b4aee02b1f0de3ba493b1d47135042d9c0/src/utils/actionUtils.ts#L81

kevincobain2000 avatar May 14 '22 05:05 kevincobain2000

Thanks for your https://github.com/actions/cache/issues/505#issuecomment-1126833743 @markstos I didn't know that the actions/cache is going to Github's cloud and not restrcited to GHES. That'd be a bummer since we have internal code in form of packages on internal pkg managers such as satis/artifactory that we wouldn't want to publish over the cloud.

kevincobain2000 avatar May 16 '22 00:05 kevincobain2000