dagger
dagger copied to clipboard
raystack
Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data.
In the case of Kafka sink, throw Exception when none of the output proto columns are selected. Current behaviour: Empty messages get published to Kafka. Expected behaviour: Dagger should throw...
Flink's SQL interface currently doesn't support Count Windows. Add a transformer that should be able to do generic Count Window Aggregations.
Add support for Distributed tracing. Trace should be able to correlate the input and output messages in the case of Data aggregation as well.
Add Integration test for Longbow+ similar to the tests present [here](https://github.com/odpf/dagger/tree/main/dagger-tests/src/integrationtest/java/io/odpf/dagger/integrationtest). Use a Bigtable emulator in order to interact with Bigtable.
Add Integration test for Longbow similar to the tests present [here](https://github.com/odpf/dagger/tree/main/dagger-tests/src/integrationtest/java/io/odpf/dagger/integrationtest). Use a Bigtable emulator in order to interact with Bigtable.
Dagger currently doesn’t support SerDe for `Struct` and `repeated Struct` data types. Current behaviour: It sets the struct fields as null in case of deserialisation and sets empty Object for...
Dagger currently only supports serialization for Map. Add support for handling other data types.
Currently, both Kafka and Influx sink logs only the data(Row) that is being sent. Add support for logging column names as well along with data points similar to the implementation...
Add Redis as an external post-processor in order to support data enrichment.
Currently, Dagger only supports SerDe for Protobuf encoded messages. * Avro SerDe is already supported in Flink. Add support for enabling it. * Apart from SerDe, postprocessors, preprocessors, some functions...