ml-commons icon indicating copy to clipboard operation
ml-commons copied to clipboard

[FEATURE] Validate Model Update/Delete When it's In Use

Open bzhangam opened this issue 9 months ago • 8 comments

Is your feature request related to a problem? Currently, ML Commons allows updating or deleting models without validation, which can lead to the following issues:

  1. Deleting a model that is still in use, potentially breaking dependent processes.
  2. Updating critical model configurations (e.g., embedding_dimension for a dense model), which may cause incompatibility with existing index configurations or stored embeddings.

What solution would you like? To prevent these issues, we need a mechanism that tracks which resources are using a model and enforces validation before updates or deletions.

Model Deletion Rules Before deleting a model:

  1. Check if the model is currently in use.
  2. If the model is in use, ensure it is properly unregistered and no longer referenced before proceeding with deletion.

Model Update Rules Before updating a model:

  1. Validate compatibility with existing usage.
  2. If the update is incompatible, either:
    • Adjust dependencies to maintain compatibility.
    • Stop using the model before applying the update.

What alternatives have you considered? N/A

Do you have any additional context? N/A

bzhangam avatar Feb 11 '25 22:02 bzhangam