beam

beam copied to clipboard

[Failing Test]: Python Precommit failing due to RRIO test failures

1

### What happened? Python PreCommit [1] and Coverage [2] test suites are failing with the following error: ``` ERROR apache_beam.io.requestresponse_it_test:requestresponse_it_test.py:304 Unable to start redis container for RRIO tests. ``` [1]...

ahmedabu98

[Python] Disable soft delete policy when creating new default bucket.

3

addresses #31330 in Python SDK. The logic is similar to #31324.

shunping

Clear side input caches of dofninvoker when finishing bundle. Add microbenchmarks for map_fn with side inputs.

13

Improve the map_fn test by using common utilities. Add support to benchmarks to add profiling. I was originally trying to optimize the code other ways but most of the overhead...

scwhittle

Bump requests from 2.31.0 to 2.32.0 in /sdks/python/apache_beam/examples/ml-orchestration/kfp/components/preprocessing

1

Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.0. Release notes Sourced from requests's releases. v2.32.0 2.32.0 (2024-05-20) 🐍 PYCON US 2024 EDITION 🐍 Security Fixed an issue where setting verify=False on the...

dependabot[bot]

[Feature Request]: Enable withFormatRecordOnFailureFunction() equivalent for BigQuery STORAGE_WRITE_API

3

### What would you like to happen? We have a dataflow pipeline that reads data from PubSub and writes to BigQuery. Current status: BigQuery write method: STREAMING_INSERTS Function used to...

sarinasij

Updates the Java Expansion Service container to support Google ALTS authentication

1

**Please** add a meaningful description for your change here ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]...

chamikaramj

Enable to handle NotFound and AccessDenied errors in the streaming insert of `BigQueryIO`

6

Addresses https://github.com/apache/beam/issues/31226 ## Issue Summary The `BigQueryServicesImpl` of the Apache Beam SDK does not handle the errors of "Not Found" and "Permission Denied" when inserting data into BigQuery fails. This...

yu-iskw

### What happened? In https://github.com/apache/beam/commit/b33a8438ad335b75feec0c5c97e9a728795fc6ff we reverted usage of snake case globally for `TypedSchemaTransformProvider`. Creating this to track fixing this forward so that we maintain a consistent schema. One way...

chamikaramj

flink portable client configurations

6

# Descriptions Previously made an attempt to solve that in https://github.com/apache/beam/pull/28068/files, but at that time, I didn't have time to follow up. I will try to contribute better to beam...

AyWa

[Bug]: Python expansion with multiple SqlTransforms is extremely slow

7

### What happened? When building a Pipeline with multiple SqlTransforms from Beam Python, the expansion that happens in SqlTransforms is currently (Beam 2.55.0) extremely inefficient. This inefficiency has multiple sources....

gergely-g