jsii icon indicating copy to clipboard operation
jsii copied to clipboard

Feature request: Cache cleanup

Open TrevorBurnham opened this issue 10 months ago • 2 comments

Describe the feature

As a regular CDK user, the ~/Library/Caches/com.amazonaws.jsii dir on my Mac (aka defaultCacheRoot) has steadily grown. It currently stands at over 13GB, mostly made up of old versions of CDK dependencies.

I'd like to request some mechanism for preventing this cache from growing indefinitely.

Use Case

I should be able to use CDK without having to worry about gradually running out of disk space.

Proposed Solution

Two options, which are not mutually exclusive:

  1. Add a documented "clear cache" command to the CDK CLI that would allow users to clear the contents of the JSII cache directory (and possibly other caches as well; my ~/Library/Caches/aws-cdk-lib and ~/Library/Caches/@aws-cdk stand at a total of 3GB).
  2. Add a system for automatically removing items from the cache that have not been used in a certain amount of time (say, 30 days). For example, on each run JSII could touch files that it reads from the cache, then delete all files in the cache whose mtime is over the time limit.

Other Information

No response

Acknowledgements

  • [ ] I may be able to implement this feature request
  • [ ] This feature might incur a breaking change

CDK version used

2.178.2

Environment details (OS name and version, etc.)

macOS 15.3.1

TrevorBurnham avatar Feb 20 '25 20:02 TrevorBurnham

I like the automatic cleanup mechanism you're proposing! It should be safe against multiple processes touching the involved files at the same time.

Unlikely that we will get to implementing this ourselves any time soon, but I would welcome the change.

rix0rrr avatar Feb 24 '25 12:02 rix0rrr

I poked around the code some more and discovered that the automatic cache purge feature I described already exists! In fact, it was implemented when the cache was first introduced back in 2022: https://github.com/aws/jsii/pull/3724

So that raises the question of how the ~/Library/Caches/com.amazonaws.jsii dir on my machine managed to grow to 13GB. I asked Claude about it and it suggested that the beforeExit hook might not be getting called. It doesn't get called if the process terminates with process.exit, for example, which this package does a lot.

I'd suggest running some experiments to see if the pruneExpiredEntries function is getting called correctly when this package is used in CDK.

Claude suggested that one possible solution is to use Node's exit hook instead of beforeExit, since that gets called even on process.exit.

TrevorBurnham avatar Apr 26 '25 20:04 TrevorBurnham