spring-data-elasticsearch
spring-data-elasticsearch copied to clipboard
Map index name to document
We're using a date index name processor to point a document to the right index based on a document's date field value.
After saving the document we need to get it again from Elasticsearch to read some fields that are filled by an ingestion pipeline.
The issue is that after the save we don't know the index name where the document has been saved.
Could it be possible to add a @Index
annotation that maps the document's index name on the Java field?
Not clear what mean with
a date index name processor to point a document to the right index
How are these index names constructed? Can't you search with a wildcard or index aliases?
The processor points the document to an index with the structure
even if it would be possible to write the index name to the document during writing, you would not have it back in the application. And even if so - if storing an entity in index-a and returning the information as result of the save or storing it in the entity on save would work - how would you later have this information to get the entity? That would mean to store the combination of entity id and index somewhere else.
And how would an @Index
annotation help? What should be done with that?
even if it would be possible to write the index name to the document during writing, you would not have it back in the application.
I have an ingestion pipeline associated to an index, when we put a document into that index instead of being written on that specific index it gets routed to another index based on the date value contained in the said document. I've successfully done that using the RestHighLevelClient and the actual index where the document has been written is included in the response.
That would mean to store the combination of entity id and index somewhere else.
Yes, and that's where an @Index
annotation would be useful, the "_index" field of the response should be written in the annotated field of the mapped model.
For instance this is what I'm actually doing with a RestHighLevelClient to add a Model
object to index my-index
:
Model model = new Model();
Map<String, Object> map = objectMapper.convertValue(model, new TypeReference<>() {});
IndexRequest request = new IndexRequest("my-index");
request.source(map);
IndexResponse response = restHighLevelClient.index(request, RequestOptions.DEFAULT);
String index = response.getIndex();
String id = response.getId();
index
value is not "my-index", instead is "my-index-yyyy-MM-dd" where the date is based on a field contained in the Model
object.
By introducing an @Index
annotation I could add an index
field to the Model
class, annotate it and index the document by doing the following:
Model model = new Model();
Model saved = elasticsearchOperations.save(model, IndexCoordinates.of("my-index"));
After that I expect that calling saved.getIndex()
should return the same value contained in the index
variable of the previous example.
ok, I got it. Although I am not sure if @Index
would be the right name, it might lead to the misconception that one could use this field to specify where the entity should be stored, we'll probably need a different name for that.
Yes, I agree with you. Maybe @ResponseIndex
or @RetrievedIndex
could be better?
@IndexedIndexName
it is.