Issue with Elastic: Limit of mapping depth [20] has been exceeded

[Big0Bertha]

Affected module UI

Describe the bug In the UI, we encounter the error message type=search_phase_execution_exception, reason=all shards failed. After reaching out on Slack, Prajwal Pandit mentioned to follow the steps defined here. The re-indexing gave me an error (full error message below) stating Limit of mapping depth [20] has been exceeded. I searched online and it has to do with the Elastic index setting index.mapping.depth.limit I tried changing it to a higher value, but I think openmetadata overwrites it back to it's default value of 20.

To Reproduce

Add a table to openmetadata with a lot of nesting? I'm unable to debug yet which table has caused this issue...

Expected behavior That I'm able to increase the index.mapping.depth.limit to a higher value, so the indexing works.

Version:

• Running on Docker
• Elastic version: Elastic cloud
• OpenMetadata version: 1.4.6

Additional context [message]: Issues in Sink To Elastic Search. [errorSource]: Sink [failedCount]: 2 [successCount]: 98 [failedEntities]: [0]: [entity]: 98 [message]: ElasticsearchException[Elasticsearch exception [type=document_parsing_exception, reason=[1:26701] failed to parse: Limit of mapping depth [20] has been exceeded due to object field [columns.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children]]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Limit of mapping depth [20] has been exceeded due to object field [columns.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children]]]; [1]: [entity]: 99 [message]: ElasticsearchException[Elasticsearch exception [type=document_parsing_exception, reason=[1:28294] failed to parse: Limit of mapping depth [20] has been exceeded due to object field [columns.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children]]]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Limit of mapping depth [20] has been exceeded due to object field [columns.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children.children]]]; [submittedCount]: 100