beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Adding SchemaTransform translation and tests for KafkaIO
Imported from Jira [BEAM-1474](https://issues.apache.org/jira/browse/BEAM-1474). Original Jira may contain additional context. Reported by: kenn.
Imported from Jira [BEAM-1479](https://issues.apache.org/jira/browse/BEAM-1479). Original Jira may contain additional context. Reported by: kenn.
### What happened? If unbounded source is empty, watermarks are not sent from the source to downstream operators. ### Issue Priority Priority: 3 (minor) ### Issue Components - [ ]...
### What would you like to happen? This bug is part of the Beam YAML milestone. More information about Beam YAML can be found here: https://github.com/apache/beam/tree/master/sdks/python/apache_beam/yaml#beam-yaml-api A Beam YAML contribution...
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.42.2 to 1.54.3. Commits 8abec4c Release 2024-05-23 70e7095 Regenerated Clients 0b2a340 Update partitions file c1eb2d9 Update endpoints model 4c990d1 Update API model c6c1626 s3: handle unrecognized values...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps commons-cli:commons-cli from 1.6.0 to 1.8.0. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.13.8 to 1.16.21. Commits b08ae84 Release 2022-10-03 3176b00 Regenerated Clients 4e7fe92 Update endpoints model 8485699 Update API model 9fa3861 Release 2022-09-30 411f0f4 Regenerated Clients 9279164 Update endpoints...
### What would you like to happen? The Managed transform API returns `PCollectionRowTuple`s, which requires extra calls when building a pipeline. Specifically, for reads we have to call `PCollectionRowTuple.empty(pipeline).apply ......
https://github.com/apache/beam/pull/30639 added an codepath to fail early when SDK emits a large element. This PR adds a warning when we detect large elements that can be problematic, but still within...
