pinot icon indicating copy to clipboard operation
pinot copied to clipboard

Adding support to extract values from message header.

Open kishoreg opened this issue 2 years ago • 0 comments

Most stream systems have started having an envelope over the actual message. For e.g. Kafka has headers and pubsub supports attributes.

Today, we dont have a way to extract values from headers. This PR adds support for these headers.

Right now it's only supporting Kafka and leverages the existing rowmetadata implementation. I am trying to keep the PR simple. let me know if there is a better way to support this.

Will add test case shortly

Instructions:

  1. The PR has to be tagged with at least one of the following labels (*):
    1. feature
    2. bugfix
    3. performance
    4. ui
    5. backward-incompat
    6. release-notes (**)
  2. Remove these instructions before publishing the PR.

(*) Other labels to consider:

  • testing
  • dependencies
  • docker
  • kubernetes
  • observability
  • security
  • code-style
  • extension-point
  • refactor
  • cleanup

(**) Use release-notes label for scenarios like:

  • New configuration options
  • Deprecation of configurations
  • Signature changes to public methods/interfaces
  • New plugins added or old plugins removed

kishoreg avatar Jul 23 '22 18:07 kishoreg