🐛 [firestore-bigquery-export] tableRequiresUpdate always returning true

[timdrew]

Describe your configuration

• Extension name: firestore-bigquery-export
• Extension version: latest
• Configuration values (redact info where appropriate):
  • clustering: null

Describe the problem

The tableRequiresUpdate function appears to be bugged in a couple different ways depending on your config. This results in the update check always returning true and the table config being repeatedly updated unnecessarily.

Issue 1

If clustering config is null then the table will always require updates due to this faulty check:

const configCluster = JSON.stringify(config.clustering); // Evaluates to 'null'
const tableCluster = JSON.stringify(metadata.clustering?.fields || []); // Evaluates to '[]'
if (configCluster !== tableCluster) return true; // Always true

Issue 2

Within initializeRawChangeLogTable the pathParamsColExists is allowed to be undefined

const pathParamsColExists = fields.find(
        (column) => column.name === "path_params"
      ); // Evaluates to undefined if not present, rather than boolean

This is then passed into tableRequiresUpdate, and will always fail the strict equality check:

if (!!config.wildcardIds !== pathParamsColExists) return true; // Evaluates to boolean !== undefined -> always true

Steps to reproduce:

What happened? How can we make the problem occur? Supply either clustering:nullor don't have path_params column.

Expected result

The table shouldn't need updating.

Actual result

The table always updates on every run.

[timdrew]

Checking in after 3mo, any chance of a fix?

[cabljac]

Hi @timdrew

Thanks for reporting this issue. We’ve received it and are reviewing it. We'll reproduce and implement a fix as soon as we can prioritise this. We’ll provide updates as soon as possible.