FiloDB icon indicating copy to clipboard operation
FiloDB copied to clipboard

feat(downsample): exclude labels during data export

Open alextheimer opened this issue 2 years ago • 0 comments

Pull Request checklist

  • [x] The commit(s) message(s) follows the contribution guidelines ?
  • [ ] Tests for the changes have been added (for bug fixes / features) ?
  • [ ] Docs have been added / updated (for bug fixes / features) ?

Adds a configurable drop-labels sequence to export rules. If an export rule is applied to a row, any labels specified in drop-labels will be excluded from the exported data.

This PR also changes how rules are defined. Keys now apply to groups of rules; the group is iterated until an applicable rule is found. The benefit of this setup is that rule "modifiers" (i.e. drop-labels) can be specified for different sets of time-series.

      # Each row's labels are compared against all rule-group keys. If a match is found,
      #   the row's labels are compared *sequentially* against each of the group's rules until
      #   a rule meets both of the following criteria:
      #       (1) No "block" filters are matched.
      #       (2) The "allow" filters are either matched or empty.
      # This rule will be applied to the row. This means that row data is/isn't exported
      #   depending on the matched filters, and any other rule effects are also applied (such as dropped labels).
      # If a matching rule (or group) are not found, the data will not be exported.
      groups = [
        {
          key = ["ws-foo"]
          rules = [
            {
              allow-filters = [
                ["_ns_=\"my_ns\"", "bar!=\"baz\""],
                ["_ns_=\"my_other_ns\""]
              ]
              block-filters = [
                ["_metric_=~\".*bad_suffix\""]
              ]
              drop-labels = ["label1", "other_label"]
            }
          ]
        }
      ]

alextheimer avatar Oct 14 '22 07:10 alextheimer