🐛 [firestore-bigquery-export] The import script fails when run on multiple threads with millions of records.

[RyosukeNishi]

[READ] Step 1: Are you in the right place?

Issues filed here should be about bugs for a specific extension in this repository. If you have a general question, need help debugging, or fall into some other category use one of these other channels:

• For general technical questions, post a question on StackOverflow with the firebase tag.
• For general Firebase discussion, use the firebase-talk google group.
• To file a bug against the Firebase Extensions platform, or for an issue affecting multiple extensions, please reach out to Firebase support directly.

[REQUIRED] Step 2: Describe your configuration

• Extension name: firestore-bigquery-export
• Extension version:
  • The extension itself
    • 0.1.41
  • The import script
    • 0.1.19
• Configuration values (redact info where appropriate):

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

fs-bq-import-collection \
  --non-interactive \
  --project ${MY_PROJECT_NAME} \
  --source-collection-path chats/{chat_token}/messages/{message_unique_id}/reaction_groups \
  --query-collection-group true \
  --dataset firestore_export \
  --table-name-prefix reaction_groups \
  --batch-size 300 \
  --dataset-location us \
  --multi-threaded true \
  --use-new-snapshot-query-syntax true \
  --use-emulator false

What happened? How can we make the problem occur? This could be a description, log/console output, etc.

Expected result

The script succeeds.

Actual result

An error has occurred on the following documents, please re-run or insert the following query documents manually... {"startAt":{"before":true,"values":[{"referenceValue":"projects/*redacted*/databases/(default)/documents/chats/nishi-chat-token/messages/026e99f6-dd02-42c0-9d14-e5cda422d768/reaction_groups/THUMBS_UP","valueType":"referenceValue"}]},"endAt":{"before":true,"values":[{"referenceValue":"projects/*redacted*/databases/(default)/documents/chats/nishi-chat-token/messages/05a5784c-2818-45cf-84f6-1e214dcc28da/reaction_groups/THUMBS_UP","valueType":"referenceValue"}]}}

Note:

• The collection group reaction_groups has around 2.5 million documents.
• The script succeeds when I run it on a single thread.