[Feature Request]: BQ: Allow map values to be nullable in GCP SDK BigQueryIO

[assaadhjb]

What would you like to happen?

in this line : https://github.com/apache/beam/blob/60436605e1983d9bc34f5be4aec881124e22b13d/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java#L577

We enforce MAP types to have required values, this is a huge setback, especially that in previous versions it was not required, for users who would like to produce messages with optional values in their MAP types. PR where it was added: https://github.com/apache/beam/pull/33389

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• [ ] Component: Python SDK
• [x] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam YAML
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [ ] Component: Infrastructure
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[claudevdm]

@RustedBones @Abacn can you take a look at this please?

[Abacn]

ack. @assaadhjb if you already have a working example (code snippet) to demonstrate this regression could you please share.

[assaadhjb]

@Abacn it is hard to provide a snippet because what we had was a failure at BigQuery level, trying to write from an Avro file that had such not supported type. Our fix consisted on falling back on MAP with required values which is really just a workaround