beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
Fix #35081. Refer to https://s.apache.org/beam-protobuf. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Mention the appropriate issue in...
### What happened? Currently JMS.Write catches all JMSException and routes the input records to failedMessagesTags. Issues: - there is no way to understand why record has failed, input record doesn't...
With the addition of extended metadata for elements, we need to make code - especially user code - more robust to added metadata fields. This pull request adds `OutputBuilder` which...
The PreCommit Java is failing over 50% of the time Please visit https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java.yml?query=is%3Afailure+branch%3Amaster to see the logs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
These are currently logged at info level, and can be a little noisy: `INFO: Opened PARQUET writer for table 'apache-beam-2.66.0-SNAPSHOT.bqms_test_catalog_1749651126538_testWriteToPartitionedAndValidateWithBQQuery_new.test_table', partition [false, 482136, value_7]. Writing to path: gs://managed-iceberg-integration-tests/temp/BigQueryMetastoreCatalogIT/423913dc-66cf-4ead-bb01-20b7561a7896/bqms_test_catalog_1749651126538_testWriteToPartitionedAndValidateWithBQQuery_new.db/test_table/data/bool_field=false/datetime_hour=2025-01-01-00/str_trunc=value_7/b29551ce-cd32-4aa2-b832-14589c63b2d5_45eb5c5f-c612-413c-b8a6-ee49dd6fed94_1.parquet` `INFO: Closed...
### What happened? ``` pipeline: type: chain transforms: - type: ReadFromKafka name: ReadFromMyTopic config: format: JSON schema: type: "object" properties: value: { type: "string" } topic: test bootstrap_servers: kafka:9092 auto_offset_reset_config:...
- fixes #27363 - fixes #34270 ------------------------ I came from #34270. I found this [google doc page](https://cloud.google.com/bigquery/docs/error-messages) > SSLEOFError (EOF occurred in violation of protocol) > Python > This error...
### What happened? Follow the steps in the README: https://github.com/apache/beam-starter-typescript?tab=readme-ov-file Clone the repository, then: ``` ✗ npm install up to date in 402ms 27 packages are looking for funding run...
**Please** add a meaningful description for your change here ------------------------ This pull request addresses [Issue #34154](https://github.com/apache/beam/issues/34154) by extending the InsertRetryPolicy in BigQueryIO to handle non-successful (non-200) responses from BigQuery’s insertAll...
Proposing a solution to https://github.com/apache/beam/issues/26190 . It appears the Flatten was not setting a watermark, which caused following steps not to execute. The issue was returning errors on beam versions...
