Easily compare nested structured data in Discover table view

[michaelruigrok]

Let's say my documents have a common nested object (e.g. labels like foo.baz, foo.nup, foo.kay etc) that I want to compare between documents. If a document has a lot of data, the table's default source column may not display these values. Currently, this means you have to use the 'Inspect document details' pop-out view, as shown here:

The problem with this is that you can only see the values of one document at a time. To compare two documents, you need to click the inspect details button, scroll down to the relevant section, and then do the same for the next one and the next one. If you are trying to search through many documents for one with an unusual set of fields in that object, the process is very painful.

I propose having a column option that lets you toggle a column to display a full multi-line pretty JSON version of a field. This would let you quickly compare a nested object in many documents at once, without needing to click each row to open an alternative view.

Here's a rough mock-up of what this feature might look like.

I believe this is a critical feature for exploring the possible fields of dynamic documents. Since you don't know what specific fields exist, being able to drill down in a structured way helps find seldom-used fields and build understanding of how fields relate to one another.

This issue is a duplicate of #5459, which I was unable to re-open.

Additional Suggestions

Another benefit of the JSON view is making it easy to display arrays of objects. If you have object.array.obj.obj.value, and a column for object.obj.obj.value, the column could simply map the object.array accordingly and display the result as a JSON array .

for instance consider a document containing the following data:

    "items": [
      {
        "create": {
          "error": {
            "type": "mapper_parsing_exception",
            "reason": "failed to parse field [params] of type [flat_object] in document with id '3f_aDY8BcM9u71R9O4jG'. Preview of field's value: 'null'",
            "caused_by": {
              "reason": "String index out of range: -6",
              "type": "string_index_out_of_bounds_exception"
            }
          },
          "_index": "index-1",
          "_id": "3f_aDY8BcM9u71R9O4jG",
          "status": 400
        }
      },
     {
        "create": {
          "_index": "index-1",
          "_id": "WLBEDY8BnusCfASEi3d8",
          "status": 400,
          "error": {
            "reason": "some other error",
            "type": "mapper_parsing_exception"
          }
        }
      },
      {
        "create": {
          "status": 201,
          "_index": "index-1",
          "_id": "3v_aDY8BcM9u71R9O4jG",
          "_version": 1,
          "result": "created",
          "_primary_term": 4,
          "_shards": {
            "successful": 2,
            "failed": 0,
            "total": 2
          },
          "_seq_no": 1216245
        }
      }
      ]

A JSON column in the document display showing the value items.create.error could display the following:

[
{
	"type": "mapper_parsing_exception",
	"reason": "failed to parse field [params] of type [flat_object] in document with id '3f_aDY8BcM9u71R9O4jG'. Preview of field's value: 'null'",
	"caused_by": {
		"reason": "String index out of range: -6",
		"type": "string_index_out_of_bounds_exception"
	}
},
{
	"reason": "some other error",
	"type": "mapper_parsing_exception"
},
null
]

[Manuelraa]

OpenSearch Dashboard does not seem to support filtering and correctly viewing flat_objects. Would be cool to get this implemented!

[rahul-sophos]

For my use case, I found a workaround for a better view in the Discover tab by defining a scripted field as:

if (params._source.containsKey('key_at_level1') && params._source['key_at_level1'].containsKey('key_at_level2') && params._source['key_at_level1']['key_at_level2'].containsKey('key_at_level3')) { return params._source['key_at_level1']['key_at_level2']['key_at_level3']; }