beam

beam copied to clipboard

The LoadTests Java PubsubIO is failing over 50% of the time. Please visit https://github.com/apache/beam/actions/workflows/beam_LoadTests_Java_PubsubIO.yml?query=is%3Afailure+branch%3Amaster to see all failed workflow runs. See also Grafana statistics: http://metrics.beam.apache.org/d/CTYdoxP4z/ga-post-commits-status?orgId=1&viewPanel=6&var-Workflow=LoadTests%20Java%20PubsubIO

github-actions[bot]

Getting error while using JdbcIO.ReadWithPartitions

1

I want to read Customer table from MySQL and store the records in CSV file. Very simple and straightforward. Want to read 4 million records in partition. I am using...

umeshPathak

feat: Add option to control resource cleanup failure for IT

1

Adds a `failOnCleanup` parameter to the `cleanResources` method in `ResourceManagerUtils`, allowing callers to specify whether an exception should be thrown when resource cleanup fails. This provides more control over the...

liferoad

### What needs to happen? GoogleContainerTools/kaniko was recently publicly archived. https://github.com/GoogleContainerTools/kaniko/pull/3502 I remembered that Apache Beam uses kaniko for its SDK builds, so I checked the repository and confirmed it...

RyuSA

[Bug]: Can't build and deploy dataflow-runner with readFromPubSubWithAttributes (but can with readFromPubSub)

3

### What happened? I have a trivial example in Typescript, reading from a PubSub subscription and just logging output. It works fine with `readFromPubSub`, e.g. ``` const messages = await...

denis-haskin-elemental

[Dataflow Streaming] Micro Optimizations

2

### What needs to happen? Issue to tie together changes to micro optimize dataflow streaming code path ### Issue Priority Priority: 2 (default / most normal work should be filed...

arunpandianp

[Feature Request]: Add the ability to cancel timers in methods annotated with @onWindowExpiration

1

### What would you like to happen? Stateful DoFns which use looping timers in ProcessTime domain need to clear the timers in order for the pipeline to drain properly when...

slilichenko

Update MongoDB driver to mongodb-driver-legacy:5.4.0

15

Upgrade MongoDB Driver to latest available release. Note that as of 4.0 the mongo-java-driver uber jar is no longer published, so using mongodb-driver-legacy (which will pull in transitive dependencies) in...

jyemin

Add Git commit SHA to artifacts using buildnumber-maven-plugin (#18227)

9

This pull request introduces the `buildnumber-maven-plugin` to the Apache Beam project to include the Git commit SHA in the `MANIFEST.MF` file of the generated artifacts. This update helps in identifying...

gayatrikate04

added timeout for Direct_Read with Bigquery IO

1

**Please** add a meaningful description for your change here ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]...

liferoad