beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Performance change found in the test: `test_cloudml_benchmark_criteo_10GB-runtime_sec` for the metric: `runtime_sec`. For more information on how to triage the alerts, please look at `Triage performance alert issues` section of the...
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
### What happened? **Description:** When attempting to execute a SQL query on a PCollection named 'data1', the following error occurs: ``` org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.sql.validate.SqlValidatorException: Object 'data1' not found ``` **Steps to Reproduce:**...
The current message doesn't display the current value, and doesn't imply that users can have control over it.
This PR adds Load tests for PubSubIO. There are several configurations depending on the amount of data and the runner where the tests will be executed: - **local**: Data volume...
Implementing a script that generates wrappers for external SchemaTransforms, according to Option #3 in the following design doc: https://s.apache.org/autogen-wrappers The script's workflow takes place in setup.py, which can be invoked...
This PR implements Redis caching support for `RequestResponseIO` and integrates it with [Enrichment transform](https://s.apache.org/enrichment-transform). It also enables the test for RRIO and Enrichment with [HTTP Mock API](https://github.com/apache/beam/tree/master/.test-infra/mock-apis). Part of #29787...
------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description (for example:...
Throwing the error that contains the resource id could help users identify which file could cause issues. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate...
TODO: https://github.com/apache/beam/issues/30288, also mentioned in the code. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the...
Update the following metrics after we insert a batch of rows using BigQuery's InsertAll RPC. ### 1. `RowsAppendedCount` Counter Tracks the status of BigQuery Rows after the batch of InsertAll...
