beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
The PostCommit Java Dataflow V2 is failing over 50% of the time Please visit https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_DataflowV2.yml?query=is%3Afailure+branch%3Amaster to see the logs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
Bumps [pymysql](https://github.com/PyMySQL/PyMySQL) from 1.1.0 to 1.1.1. Release notes Sourced from pymysql's releases. v1.1.1 [!WARNING] This release fixes a vulnerability (CVE-2024-36039). All users are recommended to update to this version. If...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
### What would you like to happen? Allow `BigTableIO.ReadChangeStream` to have a `withoutValidation` option similar to https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java#L662, and make sure to not check app profile related properties ### Issue Priority...
The PostRelease Nightly Snapshot is failing over 50% of the time Please visit https://github.com/apache/beam/actions/workflows/beam_PostRelease_NightlySnapshot.yml?query=is%3Afailure+branch%3Amaster to see the logs.
### What needs to happen? Beam portable schemas include primitive and more complex types (represented as [logical types](https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/logicaltypes)). Some of these types are supported in the Python SDK: https://github.com/apache/beam/blob/99202b237e364bf77f40b6da0ec22cb7b17c37d0/sdks/python/apache_beam/typehints/schemas.py#L23-L41 When...
Integrate direct path/fan out logic ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue...
**Please** add a meaningful description for your change here fixed #31724 This PR Contains implementation `equals` and `hashCode` of `FlinkOrderedListState` ------------------------ Thank you for your contribution! Follow this checklist to...
The current IcebergIO implementation supports only a fixed set of catalog properties. This is an unnecessary limitation for users and in some cases makes the connector unusable (e.g. if an...
Controls min and max number of connections to connection pool. For more details on this BigQuery Storage Write API feature, see https://cloud.google.com/bigquery/docs/write-api-best-practices#connection_pool_management
### What happened? We are currently executing a dataflow operation to transfer data from Kafka to BigQuery. Within this data flow, we have established a predetermined schema for BigQuery, which...
