Ingestion-time partitioning not (fully) supported

[fantastisch]

What happened?

When trying the emulator, I came across a limitation for using ingestion-time partitioned tables. It seems the emulator does not create the _PARTITIONTIME and _PARTITIONDATE columns.

When I create an ingestion-time partitioned table, and try to insert rows (using the Streaming Insert API) directly to a partition (by appending $20240517), a BigQueryException is returned:

com.google.cloud.bigquery.BigQueryException: table some_table$20240517 is not found

When trying to run a query against an ingestion-time partitioned table that references the _PARTITIONTIME column, anothger exception is thrown:

com.google.cloud.bigquery.BigQueryException: failed to analyze: INVALID_ARGUMENT: Unrecognized name: _PARTITIONTIME [at 1:8]

What did you expect to happen?

• the _PARTITIONTIME and _PARTITIONDATE columns should be created for ingestion-time partitioned tables.
• the _PARTITIONTIME and _PARTITIONDATE columns should be filled based on the current time OR the specified partition
• the _PARTITIONTIME and _PARTITIONDATE columns should be usable in queries

How can we reproduce it (as minimally and precisely as possible)?

• create an ingestion-time partitioned table
• insert a row directly into a partition (using table$YYYYmmdd) as destination table
• running a query against the table should return a value for the _PARTITIONTIME field

Anything else we need to know?

No response

[ohaibbq]

Thanks for the report! This can be merged with #152