automatic
automatic copied to clipboard
[Feature]: Add Option To Keep Checkpoints In VRAM
Feature description
For those with extra VRAM, it would be nice to be able to keep some checkpoints in VRAM to be able to serve multiple models from the server without incurring the model load cost (I'm aware there's a RAM cache option but that still adds a cost)
Version Platform Description
No response