extensions icon indicating copy to clipboard operation
extensions copied to clipboard

[firestore-bigquery-export] not creating partitioned table

Open soarb opened this issue 2 years ago • 0 comments

[REQUIRED] Step 2: Describe your configuration

  • Extension name: firestore-bigquery-export
  • Extension version: `0.1.24
  • Configuration values (redact info where appropriate):
    • Cloud functions location: europe-west-1
    • BigQuery Dataset location: eu
    • Collection path: sales
    • Dataset ID: firestore_stream_v2
    • Table ID: sales_by_month
    • BigQuery SQL table Time Partitioning option type: MONTH
    • BigQuery Time Partitioning column name: creation_date
    • Firestore Document field name for BigQuery SQL Time Partitioning field: created
    • BigQuery SQL Time Partitioning table schema field(column) type: TIMESTAMP

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

Configure the extension with ANY combination of partitioning options. Simply specifying the BigQuery SQL table Time Partitioning option type appears to have no effect.

Expected result

After running the script to backfill the table from my Firestore collection I expect to see a partitioned table by MONTH using the 'created' Timestamp field from each 'sales' document ingested.

Actual result

Always the same result no matter what combination of options I supply. A table with the following schema:-

image

... with the following details:-

image

Here's an example of the additional details I'd expect to see in the table created:-

New Project

I raised this issue #621 which has since been closed. Did it ever make it into the current release?

soarb avatar Jul 22 '22 13:07 soarb