beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Performance change found in the test: `cogbk_python_batch_load_test_reiterate_4times_2MB_values` for the metric: `runtime`. For more information on how to triage the alerts, please look at `Triage performance alert issues` section of the...
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
This PR was created by automation - by default this does not execute tests due to a limitation in GitHub Actions. To review this PR, please (1) review the contents...
### What happened? The following pipeline will fail ``` pipeline: transforms: - type: Create name: Source config: elements: - id: 1 input: Source - type: LogForTesting input: Source ``` with...
Split this notebook addition separately from the blog post "Unit Testing in Beam" (will reference it there). ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate...
### What happened? An edge case leading to data corruption: For StorageApiWriteShardedREcords, We maintain a client pool via a static Map of key as DestinationT type: [1] If there are...
### What would you like to happen? We added the warnings in Python SDK: https://github.com/apache/beam/issues/31607#issuecomment-2169043225 but to my knowledge, we don't have them in Java or Go, which might result...
### What happened? When specifying explicit inputs to transforms in Beam YAML, the error message is difficult to decipher. For example, running the following pipeline: ``` pipeline: type: chain transforms:...
### What happened? Beam YAML's PyTransform does not allow a caller to specify `error_handling` similar to other turnkey transforms. One would expect the following to work: ``` - type: PyTransform...
### What would you like to happen? I'd like to add a native Java connector to write messages to [Solace](https://solace.com/) message broker. See the design doc for details: https://docs.google.com/document/d/1mPQFT4OCEUjFIoEwkL3W4mQVj0JuCC0TXmYt-8CBV-E/edit?usp=sharing ###...
The PerformanceTests Kafka IO is failing over 50% of the time. Please visit https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_Kafka_IO.yml?query=is%3Afailure+branch%3Amaster to see all failed workflow runs. See also Grafana statistics: http://metrics.beam.apache.org/d/CTYdoxP4z/ga-post-commits-status?orgId=1&viewPanel=6&var-Workflow=PerformanceTests%20Kafka%20IO