beam
beam copied to clipboard
apache
Apache Beam is a unified programming model for Batch and Streaming data processing.
### What happened? I use SqlTransform component in an Apache Beam pipeline running in DataFlow. If I add one more variable in the SQL query, I get the error: `...
### What happened? Starting of Beam version `2.52.0`, for some reason the jdbc library imported is the thin version, and not the `all` version. Given this `pom.xml` file: ```xml 4.0.0...
### What happened? `org.apache.beam.runners.core.metrics.MetricUpdates.isEmpty` method does not check for `gaugeUpdates` to be empty. ### Issue Priority Priority: 2 (default / most bugs should be filed as P2) ### Issue Components...
To make a PTransform upgradable, we need to write some translation logic for it (see [TransformPayloadTranslator](https://github.com/apache/beam/blob/d560aa63d58145546def09aba7db6f0c8f39dbe0/sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/PTransformTranslation.java#L631)). Essentially, this is logic that can: - translate a transform to its corresponding FunctionSpec...
Bumps com.gradle.common-custom-user-data-gradle-plugin from 2.0.1 to 2.0.2. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Reimplement throttling counters after gcsio migration to google-cloud-storage client library. The previous implementation is at https://github.com/apache/beam/blob/bb044f4863cd151214f61174ac3c097bea724098/sdks/python/apache_beam/io/gcp/gcsio_overrides.py#L54
There was a prior email dev thread about PubSub ordering key[1] but that was in reference to reads. It should be possible to support pubsub Write with ordering key. We...
### What would you like to happen? This is a follow-up FR of #31330 to add warning if temp location bucket has soft delete enabled for Go SDK. Java and...
### What would you like to happen? This is a follow-up FR of #31330 to disable soft delete policy in default bucket creation for Go SDK. Java and Python implementation...