Dynamic JSON column with `.` (dots) in keys is interpreted incorrectly

[eranikid]

Describe the bug

When using K2Bridge with an Azure Data Explorer (ADX) data source, filtering on fields within a dynamic column fails if the field key contains dots (e.g., k8s.pod.name). This issue is particularly relevant when working with OpenTelemetry (OTEL) logs, as the semantic conventions recommend using dot-separated names for attributes.

While Kibana correctly discovers and displays these fields in the UI, any attempt to filter on them results in an empty result set. We suspect this is because the query generated by Kibana (e.g., for ResourceAttributes.k8s.container.name) is being incorrectly translated into a KQL query that searches for a nested JSON object (ResourceAttributes["k8s"]["container"]["name"]) instead of accessing the key with a literal dot in its name (ResourceAttributes["k8s.container.name"]).

---

To Reproduce

1. Set up an ADX table with a dynamic column. In our case, the column is named ResourceAttributes.

2. Ingest data into this table where the dynamic column contains JSON with keys using dot notation. For example:

   {
   	"deployment.environment.name": "l",
   	"telemetry.collector.name": "alloy-logs",
   	"k8s.container.name": "web",
   	"k8s.pod.ip": "8.8.8.8"
   }
   

3. Configure K2Bridge to connect to this ADX cluster and table.

4. In Kibana, create an index pattern for the table.

5. Navigate to the Discover view. Observe that the fields with dot notation are correctly parsed and displayed as separate fields.

6. Click the "Filter for value" icon next to one of these fields (e.g., for the value web on the ResourceAttributes.k8s.container.name field).

---

Expected Behavior

A filter should be successfully applied, and the results should be narrowed down to show only the documents that match the selected value.

---

Actual Behavior

The query returns no results, and Kibana displays the message: "No results match your search criteria".

---

Additional Context: Attempted Workaround

We attempted a workaround by creating a KQL function view that replaces dots (.) with underscores (_) in the keys of the ResourceAttributes column.

.create function Logs_Fixed() {
  Logs
  | extend ResourceAttributes = bag_pack(
      'k8s_container_name', ResourceAttributes['k8s.container.name'],
      'k8s_pod_ip', ResourceAttributes['k8s.pod.ip'],
      // ... and so on for all keys
    )
}

However, when a new Kibana index pattern was pointed to this function view, a new issue arose: all dynamic columns were unexpectedly interpreted as type string by Kibana. This made the fields completely unusable for filtering or visualizations, defeating the purpose of the workaround.