signoz experiment: custom running difference function for rate

experiment: custom running difference function for rate

Open srikanthccv opened this issue 3 years ago • 3 comments

Part of #1294

This is an experiment to use executable UDF to adjust for counter resets when calculating rate. Since this function is usually applied to limited number of points in the result I have used maximum of 10k values for comparing the response times.

Number of values	runningDiff	runningDiffCustom
- 10	~ 0.01	~ 0.03
- 50	~ 0.01	~ 0.03-0.035
- 100	~ 0.02	~ 0.03-0.05
- 200	~ 0.025	~ 0.04
- 500	~ 0.025-.03	~ 0.04
- 1000	~ 0.033	~ 0.05
- 2000	~ 0.04	~ 0.05
- 5000	~ 0.05	~ 0.07
- 10000	~ 0.065	~ 0.07

This as is not really ready be used because runningDifference function returns a table with columns but user defined function need to return type such as Array(Float64). The result type needs to be correlated to rows it was pivoted from. There appears to be a way to read the intermediary table and return processed table back https://github.com/ClickHouse/ClickHouse/blob/36d1c8238a03d2dbe29c4b872ec50bfeb63fa693/tests/integration/test_executable_table_function/test.py#L95.