[Feature Request]: Allow setting BigQuery endpoint, for example to use bigquery emulator

[faisalhasnain]

What happened?

There is no way to specify root url for bigquery service like we have for pubsub (PubsubOptions.setPubsubRootUrl). In particular i was trying leverage bigquery-emulator for my local development environment and that was the only blocker.

Issue Priority

Priority: 3 (minor)

Issue Components

• [ ] Component: Python SDK
• [X] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[pablomarti]

I am facing the same problem, I would like to be able to use the emulator. After checking the BigQueryIO class at public static <T> TypedRead<T> read(SerializableFunction<SchemaAndRecord, T> parseFn) it seems it could be possible by having a wrapper around setBigQueryServices?

[myhau]

I would also like to use BigQuery emulator in our local test suite, but overriding base url in BigQueryIO does not seem to be supported.

I'm happy to add support for this, but wanted to know maintainers' opinion first.

What do you think? cc: @kennknowles @ahmedabu98

Btw issue https://github.com/apache/beam/issues/28841 seems to be a duplicate of this one.

[kennknowles]

I think this is a fine idea.

[kberezin-nshl]

.take-issue

[tomaslink]

I am facing the same problem with Python. I see the PR added this feature only for Java. Should we create another separate Issue for this?

[Abacn]

feature request for Python SDK is tracked in #23598