beam
beam copied to clipboard
[Feature Request]: Add with .withFormatFunction() equivalent to Python SDK BigQueryIO
What would you like to happen?
Support customization for dynamic destinations when using Python SDK, BigQueryIO to allow users to embed destination tables without writing it to BigQuery.
A current workaround would to add destionation_table_name as a dummy column in the TableRow object and then use then set ignore_unkown_columns=True and then pass a callable table=getTableDestinationFn that gets called on each element and returns the destination_table_name.
Issue Priority
Priority: 3 (nice-to-have improvement)
Issue Components
- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner