beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
### What happened? I try running this script: [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount_xlang.py](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount_xlang.py) `$ python wordcount_xlang.py --output ./ --expansion_service_jar beam-sdks-java-io-expansion-service-2.55.0.jar` I downloaded the JAR file from: [https://repo1.maven.org/maven2/org/apache/beam/beam-sdks-java-io-expansion-service/2.55.0/](https://repo1.maven.org/maven2/org/apache/beam/beam-sdks-java-io-expansion-service/2.55.0/) Here is the output: ``` $ python...
A small optimization for `SortValues` transform to avoid doubly roundtrip encoding values that are already `byte[]`s -- they can be passed directly to the Sorter. ------------------------ Thank you for your...
### What needs to happen? Add support for Python 3.12 ### Issue Priority Priority: 2 (default / most normal work should be filed as P2) ### Issue Components - [X]...
### What happened? The Python SDK's [avro_type_to_beam_type](https://github.com/apache/beam/blob/4d350cae2a92b40787304d3c80e37b201bd99cc0/sdks/python/apache_beam/io/avroio.py#L552) function maps all Union types to: ``` type { nullable: true logical_type { urn: "beam:logical:pythonsdk_any:v1" } ``` which results in this exception: ```...
Since there is a continious demand for it, just an overview for current state of Support for Apache Pulsar: - when looking into beams i/o connector matrix **pulsar is not...
### What happened? When there are idle subtasks in flink, they dont propagate watermarks to downstream operators and thus windowing function that are based on watermarks never get triggered. I...
Addresses https://github.com/apache/beam/issues/31007 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description...
### What happened? I am using Tensorflow Transform with Flink as the Beam runner, and the versions I am using are Beam 2.48.0 and Flink 1.16.2. After adding too many...
### What would you like to happen? Current state of pulsar support is not available in doc/ beams i/o connector matrix on https://beam.apache.org/documentation/io/connectors/ would be great to add it. Overview...
Remove usages of a deprecated test helper assertDictContainsSubset in apache/beam. fixes #31048 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:...
