beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Fixes https://github.com/apache/beam/issues/29099 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description...
Include PartitionStartRecord, PartitionEndRecord, PartitionEventRecord, MoveInEvent, MoveOutEvent. These will be used in the following PRs - the returned change stream record will be parsed into these models, and corresponding actions will...
Adds Async Dofns and their tests to BEAM. Meant to be used in cases where processing is long and we don't want to block the GetWork/CommitWork streams waiting for the...
This PR has the purpose of implementing a GCP resource access control using a system managed through Git and Terraform The idea is to have 5 roles: - Beam Viewer...
Changes the default Python runner to be prism (when prism is supported). Excludes it from interactive environments for now. TODO: Wait for next release cut so this has time to...
https://github.com/apache/beam/blob/75cf7e182c9af656db147050f2cab1b7b374ee97/sdks/go/pkg/beam/core/runtime/xlangx/expansionx/download.go#L142-L164 To fix the issue, we need to validate the file paths extracted from the zip archive to ensure they do not contain directory traversal elements (`..`) and are confined...
### What needs to happen? Just add a new subclass of [IcebergCatalogBaseIT](https://github.com/apache/beam/blob/master/sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/catalog/IcebergCatalogBaseIT.java) for REST catalog tests ### Issue Priority Priority: 2 (default / most normal work should be filed as...
This is ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description...
### What needs to happen? I was going through https://cloud.google.com/dataflow/docs/notebooks/alloydb_product_catalog_embeddings and couldn't quite get my DB set up correctly - I was getting: ``` An SQLAlchemy error occurred during setup:...
### What needs to happen? KafkaIO support has been extended to the Managed Service for Apache Kafka. Due to this, features require tests that can utilize clusters and topics within...
