beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
This rolls forward the Java SDK reference implementation of `Redistribute`. This does not include runner translations. - In the prior attempt at this change: many runners failed the Redistribute tests...
Addresses #25676. Draft PR to maintain commit history. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention...
Add work processing context R: @scwhittle ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate...
Bumps [pydantic](https://github.com/pydantic/pydantic) from 1.10.2 to 1.10.13. Release notes Sourced from pydantic's releases. V1.10.13 2023-09-27 What's Changed Update pip commands to install 1.10 by @chbndrhnns in pydantic/pydantic#6930 Make the v1 mypy...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
# Issue When `org.elasticsearch.client.ResponseException` is thrown in `org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.BulkIO.BulkIOBaseFn#flushBatch`, it throws as is, which will cause infinite retries in the pipeline. Example of a `org.elasticsearch.client.ResponseException` that is not handled gracefully before...
### What happened? I'm attempting to use early triggering and `PaneInfo` to limit bundle sizes to avoid running into the dataflow limit of 80MB and have found that PaneInfo does...
### What happened? # Test code ```python import apache_beam as beam import datetime CSV_DATA = r"""a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r 1,"text",1,21,5945023,376974,0,0,0,1,2,0,4,,,,,, """ filename = '/tmp/input.csv' with open(filename, 'w') as f: f.write(CSV_DATA) # less than...
### What needs to happen? GCR is deprecated in favor of Google Artifact registry. We use GCR in some integration tests, such as Python ValidatesContainer test, and possibly a few...
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 25.0.5+incompatible to 26.1.1+incompatible. Release notes Sourced from github.com/docker/docker's releases. v26.1.1 26.1.1 For a full list of pull requests and changes in this release, refer to the relevant...
**Please** add a meaningful description for your change here Fixes #31139 by changing the subclasses we iterate to recursively get all subclasses of PipelineOptions. cc @rohandhoopar-glean ------------------------ Thank you for...
