horaedb icon indicating copy to clipboard operation
horaedb copied to clipboard

Add downsapmling for ceresdb

Open chenmudu opened this issue 3 years ago • 2 comments

Describe This Problem

At this stage, downsampling has become one of the basic capabilities in the field of time-series databases, which can solve problems such as excessive data volume returns under certain special circumstances (large time interval, high data set density).

CeresDB should support such a capability, at least in the immediate plans.

Proposal

The engine layer supports downsampling, and supports dynamic downsampling and configuration downsampling. Matching, clients in each language should also support such an API.

Notice:

  • dynamic downsampling:Support can automatically adjust the sampling granularity according to the time interval.
  • configuration downsampling:Rely on the user to provide the accuracy of the downsampling.

Additional Context

OpenTSDB Downsampling

chenmudu avatar Mar 02 '23 07:03 chenmudu

Thanks for suggestion.

Downsample is indeed important for time-series user-case, to support downsample an RFC maybe required to discuss

  • pros and cons of downsample
  • when query will use downsampled data
  • how to run downsample job(in main server or some other background components).
  • ....

jiacai2050 avatar Mar 02 '23 08:03 jiacai2050

  • pros and cons of downsample

In most people's opinion, downsampling should have no disadvantages for time series data. Maybe I'm missing something, can anyone add here.

chenmudu avatar Mar 02 '23 09:03 chenmudu