beam icon indicating copy to clipboard operation
beam copied to clipboard

[Bug]: Python SDK chain after WriteToBigQuery example is incorrect for file_loads method

Open tangsiwei0831 opened this issue 1 year ago • 0 comments

What happened?

In https://github.com/apache/beam/blob/611676d108b26ee378a2b0c128c855017d162772/sdks/python/apache_beam/io/gcp/bigquery.py#L286, the documentation states that the chain operation for file loads is return (result.load_jobid_pairs, result.copy_jobid_pairs) | beam.Flatten()

In fact, according to https://github.com/apache/beam/blob/611676d108b26ee378a2b0c128c855017d162772/sdks/python/apache_beam/io/gcp/bigquery.py#L2230 and https://github.com/apache/beam/blob/611676d108b26ee378a2b0c128c855017d162772/sdks/python/apache_beam/io/gcp/bigquery.py#L2234, it should be return (result.destination_load_jobid_pairs, result.destination_copy_jobid_pairs) | beam.Flatten()

Issue Priority

Priority: 3 (minor)

Issue Components

  • [X] Component: Python SDK
  • [ ] Component: Java SDK
  • [ ] Component: Go SDK
  • [ ] Component: Typescript SDK
  • [ ] Component: IO connector
  • [ ] Component: Beam YAML
  • [X] Component: Beam examples
  • [ ] Component: Beam playground
  • [ ] Component: Beam katas
  • [X] Component: Website
  • [ ] Component: Spark Runner
  • [ ] Component: Flink Runner
  • [ ] Component: Samza Runner
  • [ ] Component: Twister2 Runner
  • [ ] Component: Hazelcast Jet Runner
  • [ ] Component: Google Cloud Dataflow Runner

tangsiwei0831 avatar May 22 '24 01:05 tangsiwei0831