InvokeAI icon indicating copy to clipboard operation
InvokeAI copied to clipboard

[MM] add API routes for getting & setting MM cache sizes

Open lstein opened this issue 1 year ago • 1 comments

Summary

This PR adds the following model manager-related API routes:

  • get_cache_size Fetch the maximum size of the RAM or VRAM cache.
  • set_cache_size Set the maximum size of the RAM or VRAM cache, optionally making the setting persistent by updating invokeai.yaml
  • get_stats Get performance stats (cache hits, misses, high-water mark, loaded model sizes) for the RAM cache

Related Issues / Discussions

QA Instructions

  1. Fetch and set the RAM and VRAM cache sizes using the swagger UI.
  2. Check that when the persistent parameter is set to True in set_cache_size the updated values get written to the config file.
  3. After running a generation, inspect the output of get_stats.

Merge Plan

Merge when approved.

Checklist

  • [X] The PR has a short but descriptive title, suitable for a changelog
  • [X] Tests added / updated (if applicable)
  • [X] Documentation added / updated (if applicable)

lstein avatar Jun 20 '24 01:06 lstein

What's the motivation for these endpoints? Are there plans to build a UI for this?

The new cache-setting routes were suggested during a discord discussion with @blessedcoolant and @psychedelicious: https://discord.com/channels/1020123559063990373/1250595668792443002/1252739211652632668

Idea is to have sliders in the settings UI.

lstein avatar Jun 21 '24 19:06 lstein