elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

[ML] Deleting a trained model can emit deprecation warnings related to ingest pipeline configs

Open davidkyle opened this issue 1 year ago • 2 comments

Elasticsearch Version

8.12

Installed Plugins

No response

Java Version

bundled

OS Version

any

Problem Description

When a trained model is delete the ingest pipeline configurations are checked to ensure the model is not used in a pipeline. If used the deletion fails and the user is instructed to use the force option.

That check on the pipeline configs causes any deprecation messages related to the pipeline config to be included in the response.

DELETE _ml/trained_models/my-model

#! The default [remove_binary] value of 'false' is deprecated and will be set to 'true' in a future release. Set [remove_binary] explicitly to 'true' or 'false' to ensure no behavior change.
{
  "acknowledged": true
}

Any pipeline processor that contains deprecated config will cause the warning, this is confusing and hard to track down the source of the problem.

Steps to Reproduce

  1. Install any trained model, downloading the builtin .elser_model_2 is the easy option.
  2. Create an ingest pipeline containing the attachment processor with deprecated config
PUT _ingest/pipeline/attachment
{
  "description" : "Extract attachment information",
  "processors" : [
    {
      "attachment" : {
        "field" : "data"
      }
    }
  ]
}
  1. Delete .elser_model_2
DELETE _ml/trained_models/.elser_model_2

And observe the deprecation waring in the response

#! The default [remove_binary] value of 'false' is deprecated and will be set to 'true' in a future release. Set [remove_binary] explicitly to 'true' or 'false' to ensure no behavior change.

Logs (if relevant)

No response

davidkyle avatar Feb 01 '24 10:02 davidkyle

Pinging @elastic/ml-core (Team:ML)

elasticsearchmachine avatar Feb 01 '24 10:02 elasticsearchmachine

This is a very similar problem to #87978. Another related problem with deleting a trained model now will be that the check will be incredibly slow if the user has 100 pipelines with complex Grok processors in them, just like we saw in #87978. It should be possible to fix this by reusing the utility function that was added in #87978 (or a very similar new one that could be added to the same InferenceProcessorInfoExtractor class).

droberts195 avatar Feb 01 '24 10:02 droberts195

Related: https://github.com/elastic/ml-team/issues/1204

dimkots avatar May 14 '24 13:05 dimkots