beam

beam copied to clipboard

Add a workaround for BEAM-20873 for optimized list side inputs.

2

This triggered a case with Dataflow Runner v1 and PTransformOverrides that exposed https://github.com/apache/beam/issues/20873 . ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly...

robertwb

add computation state cache, config loaders, and status pages

2

Add computation state cache and config loaders that load the computation state cache move status pages out of StreamingDataflowWorker file R: @scwhittle ------------------------ Thank you for your contribution! Follow this...

m-trieu

(testing, no need to review) update libraries-bom to 26.37.0

7

Update google_cloud_platform_libraries_bom to 26.38.0. Associated updates to google_cloud_spanner_version, gax_version, guava and protobuf_version. Note: libraries-bom to 26.38.0 is not available yet, raising pr to test for now with 26.37.0. ------------------------ Thank...

zhumin8

Rollback View.asList() side inputs optimizations.

1

------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description (for example:...

robertwb

### What happened? [PostCommit Java Dataflow V1](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_DataflowV1.yml) shows repeated failures, [for example](https://ge.apache.org/s/5nvp2452cisp2/console-log?anchor=232&page=1): ``` org.apache.beam.sdk.io.gcp.healthcare.FhirIOWriteIT > testFhirIO_Import[DSTU2] FAILED java.lang.NullPointerException at FhirIOWriteIT.java:194 org.apache.beam.sdk.io.gcp.healthcare.FhirIOWriteIT > testFhirIO_Import[STU3] FAILED java.lang.NullPointerException at FhirIOWriteIT.java:194 org.apache.beam.sdk.io.gcp.healthcare.FhirIOWriteIT > testFhirIO_Import[R4]...

damondouglas

Pass original message down through conversion for storage write api

13

Enable users to specify an alternate way to generate the table row for the error output for BQIO's storage write api. The user passes in a function of ElementT ->...

johnjcasey

Set backlog in gauge metric for Kafka Reader

5

There are gauges created for per partition backlog in the kafka reader, but its never actually set. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate...

Naireen

[YAML] Add the ability to pre-process yaml files with jinja2.

11

This will allow Beam YAML to be used for various template applications. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:...

robertwb

Add random suffix to IT resource IDs to avoid collision

5

There have been cases of name collisions on parallel tests, so adding random suffix to avoid collision. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate...

Polber

Suppress BigQuery read stream splitAtFraction when API busy call or timeout

4

`Resolve` https://github.com/apache/beam/issues/30646#issuecomment-2080102737 * Reject split attempt early when there is an ongoing hasNext call in work item thread * TImeout hasNext call in splitAtFraction in alignment with client splitReadStreamSettings The...

Abacn