Incorrect Elasticsearch mapping: cannot get memory `results`

[fedorovychh]

Summary

When querying memories filtered by user_id and run_id, the service returns a successful response (200) but with empty results, even though data exists in the Elasticsearch index.

Root Cause

The Elasticsearch mapping for the metadata field initializes only metadata.user_id as a keyword. After inserting data that includes a run_id, Elasticsearch dynamically creates metadata.run_id as an object instead of a keyword. As a result, subsequent queries are generated incorrectly and reference metadata.run_id.enum instead of metadata.run_id.

This causes the search query to return no results.

Details

Initial mapping created by the service:

"metadata": {
  "type": "object",
  "properties": {
    "user_id": { "type": "keyword" }
  }
}

After adding a memory with run_id, ES auto-creates:

"metadata.run_id": {
  "type": "object",
  "properties": {
    "enum": { "type": "keyword" }
  }
}

Expected behavior is to query:

metadata.run_id

Actual behavior becomes:

metadata.run_id.enum

This mismatch results in empty search responses.

Fix

Explicitly define run_id in the mapping as keyword:

"metadata": {
  "type": "object",
  "properties": {
    "user_id": { "type": "keyword" },
    "run_id": { "type": "keyword" }
  }
}

Impact

• Queries with run_id filter do not return stored memories
• Confusing behavior: data appears in ES but is not retrievable through the API
• Requires manual mapping adjustment to work correctly

Suggested Solution

Update default ES mapping to include run_id as a keyword field, similar to user_id, to avoid dynamic object creation and ensure proper query behavior.

[parthshr370]

Hey @fedorovychh thanks for reporting this , onto it rn !