beam icon indicating copy to clipboard operation
beam copied to clipboard

[Bug]: BigQueryIO performance overhead on DynamicDestinations on parsing json

Open stankiewicz opened this issue 3 weeks ago • 7 comments

What happened?

Where: https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinationsHelpers.java#L305-L314

Why: validation of static valueprovider is per element Impact: 20% overhead on this stage.

Image

Request: store outcome of validation within the object itself.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • [ ] Component: Python SDK
  • [x] Component: Java SDK
  • [ ] Component: Go SDK
  • [ ] Component: Typescript SDK
  • [x] Component: IO connector
  • [ ] Component: Beam YAML
  • [ ] Component: Beam examples
  • [ ] Component: Beam playground
  • [ ] Component: Beam katas
  • [ ] Component: Website
  • [ ] Component: Infrastructure
  • [ ] Component: Spark Runner
  • [ ] Component: Flink Runner
  • [ ] Component: Samza Runner
  • [ ] Component: Twister2 Runner
  • [ ] Component: Hazelcast Jet Runner
  • [ ] Component: Google Cloud Dataflow Runner

stankiewicz avatar Dec 04 '25 11:12 stankiewicz

.add-labels "good first issue"

stankiewicz avatar Dec 04 '25 11:12 stankiewicz

Label "good cannot be managed because it does not exist in the repo. Please check your spelling.

github-actions[bot] avatar Dec 04 '25 11:12 github-actions[bot]

.add-labels "good%20first%20issue"

stankiewicz avatar Dec 04 '25 11:12 stankiewicz

Label "good%20first%20issue" cannot be managed because it does not exist in the repo. Please check your spelling.

github-actions[bot] avatar Dec 04 '25 11:12 github-actions[bot]

.add-labels good first issue

stankiewicz avatar Dec 04 '25 11:12 stankiewicz

Label good cannot be managed because it does not exist in the repo. Please check your spelling.

github-actions[bot] avatar Dec 04 '25 11:12 github-actions[bot]

.take_issue

stankiewicz avatar Dec 05 '25 19:12 stankiewicz