beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Tests run on GHA should be queued shortly.
### What needs to happen? Create runnable examples using RequestResponseIO in Python for the Beam Website. ### Issue Priority Priority: 2 (default / most normal work should be filed as...
In my current scenario, I have an Apache Beam application that employs Debezium for capturing changes in data. Unlike the typical integration with Kafka, I am not using it in...
**Please** add a meaningful description for your change here ------------------------ fixes #32001 Merge `attempted` and `committed` `MonitoringInfo` by metric identity, prefer committed entries, and create `MetricResult` with the correct committed...
### What needs to happen? Some features are missing: - Support DDL for DATABASEs (create, use, drop) #35641 - If a user has set a DATABASE, they should be able...
Cherry-picks https://github.com/apache/beam/pull/37055 into the 2.70.0 release branch. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the...
[Bug]: Add SchemaFieldNumber annotations to Iceberg to prevent potential update compatibility issues
### What happened? Iceberg uses SchemaCoder in several locations. https://github.com/apache/beam/blob/6d24c3dec3bfab3d49c32ed2ef2fe3d0a8d803ef/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/SnapshotInfo.java#L153 https://github.com/apache/beam/blob/c236996a4550f92388f8688afae144fa402de171/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/ReadTask.java#L42 https://github.com/apache/beam/blob/c236996a4550f92388f8688afae144fa402de171/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/ReadTaskDescriptor.java#L37 Using `SchemaCoder` (as opposed to `RowCoder`, for example) can result in update compatibility issues for Dataflow Runner v2....
### What happened? For a few months now we've been dealing with an issue where messages are "disappearing" during scaling events. By that, I mean they are just simply not...
This change replaces the now unmaintained `gradle/gradle-build-action` with `gradle/actions/setup-gradle`. Also remove the unmaintained `gradle/wrapper-validation-action` w/o a replacement, assuming #36771 is going in. ------------------------ Thank you for your contribution! Follow this...
The ASF Release Policy mentions that a source release should not contain compiled code. Jar files, like the `gradle/wrapper/gradle-wrapper.jar` file, are compiled code and should not be included in source...