numaflow icon indicating copy to clipboard operation
numaflow copied to clipboard

All wire formats should be protobuf

Open vigith opened this issue 7 months ago • 0 comments

Summary

We use encoding/binary for serde. We need to standardize on a wire format that is scalable and also supports backward compatibility. encoding/binary encourages using gob or protobuf; I think protobuf has more benefits than gob because it is language agnostic and streaming is not implicit.

Use Cases

  • Backward compatibility
  • Language agnostic
  • Scalable

Affected components

### Components to change
- [ ] #1768 
- [ ] #1779 
- [ ] WAL storage serde using protobuf

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

vigith avatar Jun 28 '24 15:06 vigith