pinot
pinot copied to clipboard
Adding support to extract values from message header.
Most stream systems have started having an envelope over the actual message. For e.g. Kafka has headers and pubsub supports attributes.
Today, we dont have a way to extract values from headers. This PR adds support for these headers.
Right now it's only supporting Kafka and leverages the existing rowmetadata implementation. I am trying to keep the PR simple. let me know if there is a better way to support this.
Will add test case shortly
Instructions:
- The PR has to be tagged with at least one of the following labels (*):
-
feature
-
bugfix
-
performance
-
ui
-
backward-incompat
-
release-notes
(**)
-
- Remove these instructions before publishing the PR.
(*) Other labels to consider:
-
testing
-
dependencies
-
docker
-
kubernetes
-
observability
-
security
-
code-style
-
extension-point
-
refactor
-
cleanup
(**) Use release-notes
label for scenarios like:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed