Vigith Maurice

# Summary We need to make sure we have versioning for backward compatibility. ref https://docs.google.com/document/d/1xnjgBaF3rTN6n7Pbw5ggHeVBAxm_VbunjivsEwSwglg/edit?usp=sharing

Pause kafka consumer if the buffer is full to avoid topic rebalancing

1

# Summary If you consume a message from Kafka and it is not able to write to the next buffer because the buffer is full, we will have to tell...

# Summary Today, we have to fix constraints - [ ] only an element can be processed at a time - [ ] we create a map-stream (gRPC) for every...

Today the channel can be closed by either user or numa; we should make sure only numa can close that channel, as numa owns the channel.

Resume pipeline from a particular source-state

1

## Problem Today, we cannot start the numaflow pipeline at a particular source-state. We always start from a relative state (e.g., latest, oldest offset); we cannot say start at offset...

# Summary Today, we have to have at least a Source and a Sink. This means we need to write the message from the Source vertex to the ISB so...

The current logger doesn't support TRACE-level logging and debugging is getting trickier and harder. [zerolog](https://github.com/rs/zerolog) supports `TRACE` and is also more efficient.

- [x] #985 - [ ] introduce Used Defined Sink Transformers - [ ] Update documentation

Add a new Terraform blueprint for [Numaflow](https://numaflow.numaproj.io/) under streaming https://github.com/awslabs/data-on-eks/tree/main/streaming Ref: https://github.com/awslabs/data-on-eks/issues/382 ```[tasklist] ### Tasks ```

https://github.com/awslabs/data-on-eks/issues/382 ```[tasklist] ### Tasks - [ ] https://github.com/numaproj/numaflow/issues/1092 - [ ] https://github.com/numaproj/numaflow/issues/1486 - [ ] Documentation - [ ] Example with an end-to-end use case ```