InvokeAI
InvokeAI copied to clipboard
[MM] add API routes for getting & setting MM cache sizes
Summary
This PR adds the following model manager-related API routes:
get_cache_sizeFetch the maximum size of the RAM or VRAM cache.set_cache_sizeSet the maximum size of the RAM or VRAM cache, optionally making the setting persistent by updatinginvokeai.yamlget_statsGet performance stats (cache hits, misses, high-water mark, loaded model sizes) for the RAM cache
Related Issues / Discussions
QA Instructions
- Fetch and set the RAM and VRAM cache sizes using the swagger UI.
- Check that when the
persistentparameter is set to True inset_cache_sizethe updated values get written to the config file. - After running a generation, inspect the output of
get_stats.
Merge Plan
Merge when approved.
Checklist
- [X] The PR has a short but descriptive title, suitable for a changelog
- [X] Tests added / updated (if applicable)
- [X] Documentation added / updated (if applicable)
What's the motivation for these endpoints? Are there plans to build a UI for this?
The new cache-setting routes were suggested during a discord discussion with @blessedcoolant and @psychedelicious: https://discord.com/channels/1020123559063990373/1250595668792443002/1252739211652632668
Idea is to have sliders in the settings UI.