beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Add support for dataclasses with `kw_only=True`, and for dataclasses with `init=False` fields. This change encode the dataclass field names as well as their values in order to pass them to...
### What happened? I tried running a Dataflow job that reads messages from a Kafka topic using kafka-client 4.0.0 and my reads were silently failing. Downgrading to 3.7.0 caused reads...
### What would you like to happen? Currently, python dataclasses are decoded by passing their encoded fields by position. This behavior does not work for dataclasses that are defined with...
### What would you like to happen? Hi Beam team, I'm currently using Apache Beam 2.64.0 with kafka-clients version 3.9.0. Since Apache Kafka 4.0.0 was officially released in March 2025,...
**Please** add a meaningful description for your change here ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]...
See https://github.com/protocolbuffers/protobuf/blob/0a2f7757e38b07c2e40c5c885a7100e8c16eb40f/java/core/src/main/java/com/google/protobuf/CodedOutputStream.java#L700 for details. This change copies Protobuf's branchless implementation for varint length calculation. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly...
TODO - needs more testing and benchmarking ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the...
### What happened? This is observed in some of our test workflow. Even though it is not causing the test to fail, I believe it is only because of luck,...
This PR adds documentation for Apache Iceberg support in Beam SQL, including both new catalog management. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your...
### 🎯 Summary This PR cleans up `textio.py` to improve readability, maintainability, and consistency with the rest of the Beam codebase. ### ✨ Changes Included - Removed duplicated sections of...
