filter by parsed JSON string feature - fails when value is not a string

[ryn9]

As the title says, using the filter by parsed JSON string feature... It fails when the value is not a string (such as an integer or float number)

When a number - the generated WHERE clause should not be use JSONExtractString

[SpencerTorres]

I understand how JSONExtractString might break, but could you provide a screenshot + SQL + error message to make it easier to isolate the issue? Sounds like it's just the wrong input, but maybe we can modify the SQL to check if it's valid

[ryn9]

Given the following:

If I select "Search" for request_length, I get an error page with the following info...

Error encountered for query with inputs:

SELECT
Timestamp, ServiceNamespace, ServiceName, SeverityText, Body

ORDER BY
Timestamp DESC

WHERE
JSONExtractString (Body, 'request_length') = 1447


Error Message:
There is no supertype for types String, UInt16 because some of them are String\/FixedString\/Enum and some of them are not. 


Original Query:
SELECT
  Timestamp,
  ServiceNamespace,
  ServiceName,
  SeverityText,
  Body,
  toDateTime (Timestamp)
FROM
  MYDATABASE.TABLE
WHERE
  (
    Timestamp >= fromUnixTimestamp64Milli (1759838146000)
    AND Timestamp <= fromUnixTimestamp64Milli (1759839046000)
  )
  AND (JSONExtractString (Body, 'request_length') = 1447)
ORDER BY
  Timestamp DESC
LIMIT
  200
OFFSET
  0

NOTE: My default select is set to: "Timestamp, ServiceNamespace, ServiceName, SeverityText, Body"

[pulpdrew]

Hi @ryn9 sorry for the delay here.

1. Are you able to confirm that you are still facing the issue on the latest version of HyperDX? There have been a number of bug fixes in our JSON support in the last month or so, and I'm curious is this issue has already been fixed. I've not been able to reproduce yet.
2. If you are still facing the issue, are you able to share the schema of the table you are querying?

[ryn9]

I will aim to update to the latest this week and test again.

[ryn9]

I just tested with 2.7.1 and this is still a problem.

Note - the issue when when I expand a Body as JSON and then click the search button for one of the expanded attributes, but the attribute is a number. (like in the screenshot above).

In the screenshot above imagine I click on "Search" associated with the request_length field. It generated a search WHERE clause as JSONExtractString(Body, 'request_length') = 1447 Clickhouse does not like that as you are asking it compare a string to a number.

Basically the search button creates a "JSONExtractString" WHERE clause instead of "JSONExtractFloat" WHERE clause, even when the "Expand JSON" option in the UI has identified the field as a number.

Please let me know if I need to explain that better.

[pulpdrew]

Thank you for testing it again, and for the detailed explanation. Would you mind sharing the schema of the table that is being queried?

[ryn9]

The table....

    `ServiceNamespace` LowCardinality(String) DEFAULT '' CODEC(ZSTD(1)),
    `ServiceName` LowCardinality(String) DEFAULT '' CODEC(ZSTD(1)),
    `Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
    `LogTimestamp` DateTime64(9) DEFAULT toDateTime64(0, 9) CODEC(Delta(8), ZSTD(1)),
    `TraceId` String CODEC(ZSTD(1)),
    `SpanId` String CODEC(ZSTD(1)),
    `Flags` UInt8,
    `SeverityText` LowCardinality(String) CODEC(ZSTD(1)),
    `SeverityNumber` UInt8,
    `Body` String CODEC(ZSTD(1)),
    `ResourceSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
    `ResourceAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `ScopeSchemaUrl` LowCardinality(String) CODEC(ZSTD(1)),
    `ScopeName` String CODEC(ZSTD(1)),
    `ScopeVersion` LowCardinality(String) CODEC(ZSTD(1)),
    `ScopeAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `LogAttributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),

Note the similarities to https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/clickhouseexporter/internal/sqltemplates/logs_table.sql

[pulpdrew]

Ah thank you, that's helpful and I was able to reproduce the issue. This does appear to be a bug - thank you for reporting it! We'll try to get this fixed.