David Kyle

[ML] Remove mention of models in inference actions

2

This renames the GET, PUT and DELETE inference APIs removing the `model` parts. - `inference.delete_model` renamed to `inference.delete` - `inference.get_model` renamed to `inference.get` - `inference.put` renamed to `inference.put` The GET...

[ML] Deleting a trained model can emit deprecation warnings related to ingest pipeline configs

2

### Elasticsearch Version 8.12 ### Installed Plugins _No response_ ### Java Version _bundled_ ### OS Version any ### Problem Description When a trained model is delete the ingest pipeline configurations...

[ML] Refactor the Embedding classes

1

Extracted and generic-ised the `Embedding` and `EmbeddingChunk` classes so that the `EmbeddingRequestChunker` can handle different types of embedding.

[ML] Add start model import message

1

Adds a `[model_id] starting model import` log message and notification on model download. The `starting model import` message replaces the `imported model vocabulary` notification. Now we will see a `starting...

[ML] Wait for all shards to be active when creating the ML stats index

1

Several tests have started failing recently with the dread `no_shard_available_action_exception` when querying the `.ml-stats` index. The error occurs when an index is accessed directly after creation before any shards have...

[CI] IndexRecoveryIT testDoNotInfinitelyWaitForMapping failing

17

**Build scan:** https://gradle-enterprise.elastic.co/s/4d2lhrqhockm6/tests/:server:internalClusterTest/org.elasticsearch.indices.recovery.IndexRecoveryIT/testDoNotInfinitelyWaitForMapping **Reproduction line:** ``` ./gradlew ':server:internalClusterTest' --tests "org.elasticsearch.indices.recovery.IndexRecoveryIT.testDoNotInfinitelyWaitForMapping" -Dtests.seed=40853F21F419B395 -Dtests.jvm.argline="-Des.concurrent_search=true" -Dtests.locale=id-ID -Dtests.timezone=Asia/Jerusalem -Druntime.java=21 ``` **Applicable branches:** main **Reproduces locally?:** Didn't try **Failure history:** [Failure dashboard for `org.elasticsearch.indices.recovery.IndexRecoveryIT#testDoNotInfinitelyWaitForMapping`](https://es-delivery-stats.elastic.dev/app/dashboards#/view/dcec9e60-72ac-11ee-8f39-55975ded9e63?_g=(refreshInterval:(pause:!t,value:60000),time:(from:now-7d%2Fd,to:now))&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('testDoNotInfinitelyWaitForMapping'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.indices.recovery.IndexRecoveryIT'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium))))) **Failure...

[CI] ReplicaShardAllocatorIT testFullClusterRestartPerformNoopRecovery failing

1

**Build scan:** https://gradle-enterprise.elastic.co/s/sfl5iqhsuazz6/tests/:server:internalClusterTest/org.elasticsearch.gateway.ReplicaShardAllocatorIT/testFullClusterRestartPerformNoopRecovery **Reproduction line:** ``` ./gradlew ':server:internalClusterTest' --tests "org.elasticsearch.gateway.ReplicaShardAllocatorIT.testFullClusterRestartPerformNoopRecovery" -Dtests.seed=162905307FB74387 -Dtests.locale=nb-NO -Dtests.timezone=Etc/GMT+9 -Druntime.java=21 ``` **Applicable branches:** main **Reproduces locally?:** Didn't try **Failure history:** [Failure dashboard for `org.elasticsearch.gateway.ReplicaShardAllocatorIT#testFullClusterRestartPerformNoopRecovery`](https://es-delivery-stats.elastic.dev/app/dashboards#/view/dcec9e60-72ac-11ee-8f39-55975ded9e63?_g=(refreshInterval:(pause:!t,value:60000),time:(from:now-7d%2Fd,to:now))&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('testFullClusterRestartPerformNoopRecovery'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.gateway.ReplicaShardAllocatorIT'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium))))) **Failure excerpt:**...

[ML] Default similarity for all Cohere models and element types to Dot Product

1

non-issue as it reverts a work-around for a problem using byte embeddings

[CI] AsyncBulkByScrollActionTests testScrollResponseBatchingBehavior failing

2

**Build scan:** https://gradle-enterprise.elastic.co/s/h7gdqkimbf4tg/tests/:modules:reindex:test/org.elasticsearch.reindex.AsyncBulkByScrollActionTests/testScrollResponseBatchingBehavior **Reproduction line:** ``` ./gradlew ':modules:reindex:test' --tests "org.elasticsearch.reindex.AsyncBulkByScrollActionTests.testScrollResponseBatchingBehavior" -Dtests.seed=5638EFEF212A5042 -Dtests.locale=nl-BE -Dtests.timezone=America/Atka -Druntime.java=22 ``` **Applicable branches:** main **Reproduces locally?:** Didn't try **Failure history:** [Failure dashboard for `org.elasticsearch.reindex.AsyncBulkByScrollActionTests#testScrollResponseBatchingBehavior`](https://es-delivery-stats.elastic.dev/app/dashboards#/view/dcec9e60-72ac-11ee-8f39-55975ded9e63?_g=(refreshInterval:(pause:!t,value:60000),time:(from:now-7d%2Fd,to:now))&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('testScrollResponseBatchingBehavior'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.reindex.AsyncBulkByScrollActionTests'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium))))) **Failure excerpt:**...

[ML] Support sparse embedding models in the elasticsearch inference service

3

For a sparse embedding model created with the ml trained models APIs ``` PUT _inference/sparse_embedding/my-sparse { "service": "elasticsearch", "service_settings": { "model_id": "custom-text-expansion-model", "num_allocations": 1, "num_threads": 1 } } ```