beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
**Please** add a meaningful description for your change here ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]...
Sharded out this batch of ProcesssingTime features to complete simultaneously, once TestStream #29917 and to allow #29772 to be closed. After TestStream is in, there's a very clear hook for...
Implementing lull reporting for dataflow worker at bundle level processing. We dump a stack trace when the bundle processing time exceeds 10 mins. As part of this, we log the...
### What would you like to happen? Support customization for dynamic destinations when using Python SDK, BigQueryIO to allow users to embed destination tables without writing it to BigQuery. A...
### What happened? It appears that PreCommit timeout happens more frequently recently. the succeeded job run after the timed out run shows the following tests are added, which suggests the...
StreamingDataflowWorker is a rather large class For maintainability it would be helpful in the long run to encapsulate different pieces of the worker harness logic into their own class. ------------------------...
Performance change found in the test: `gbk_python_batch_load_test_fanout_8_times_with_2GB_10byte_records_total` for the metric: `runtime`. For more information on how to triage the alerts, please look at `Triage performance alert issues` section of the...
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
**Please** add a meaningful description for your change here ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]...
### What needs to happen? The current implementation for Beam YAML to read proto events is not allowing multiple imports from FileDescriptors. Ideally we want to support picking the right...
This is more efficient as it doesn't require an index of indices. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:...
