extensions icon indicating copy to clipboard operation
extensions copied to clipboard

Published 20 hours ago • verified publisher icon firebase

🐛 firestore-bigquery-export Unhandled error ApiError: Cannot parse response as JSON:

Open madmacc opened this issue 1 year ago • 2 comments

[REQUIRED] Step 2: Describe your configuration

  • Extension name: firestore-bigquery-export
  • Extension version: 0.1.33
  • Configuration values (redact info where appropriate): BigQuery Dataset location us-east1 Project Id myprojectid Collection path memberStatusChangeLogs Enable Wildcard Column field with Parent Firestore Document IDs (Optional) false Dataset ID mydsid Table ID memberStatusChangeLogs BigQuery SQL table Time Partitioning option type (Optional) NONE BigQuery Time Partitioning column name (Optional) Parameter not set Firestore Document field name for BigQuery SQL Time Partitioning field option (Optional) Parameter not set BigQuery SQL Time Partitioning table schema field(column) type (Optional) omit BigQuery SQL table clustering (Optional) Parameter not set Maximum number of synced documents per second (Optional) Parameter not set Backup Collection Name (Optional) Parameter not set Transform function URL (Optional) Parameter not set Use new query syntax for snapshots yes Enabled events This extension has no events enabled. If you want to enable events, reconfigure the extension.

[REQUIRED] Step 3: Describe the problem

This has been working fine for a while but on 2024-04-22 I just started getting this error: image

image

Unhandled error ApiError: Cannot parse response as JSON: <!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 502 (Server Error)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>502.</b> <ins>That’s an error.</ins>
  <p>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.  <ins>That’s all we know.</ins>

    at Util.parseHttpRespBody (/workspace/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/node_modules/@google-cloud/common/build/src/util.js:188:42)
    at Util.handleResp (/workspace/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/node_modules/@google-cloud/common/build/src/util.js:134:117)
    at /workspace/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/node_modules/@google-cloud/common/build/src/util.js:432:22
    at onResponse (/workspace/node_modules/retry-request/index.js:228:7)
    at /workspace/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/node_modules/teeny-request/build/src/index.js:233:13
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  response: PassThrough {
    _readableState: ReadableState {
… [message truncated due to size]

Steps to reproduce:

No changes to the process this just started to occur and is only occurring sometimes. Other collections running this extension seem to be ok.

Expected result

Copies the data to BQ without an error.

Actual result

Has an error and does not copy data to BQ.

madmacc avatar Apr 21 '24 21:04 madmacc

Hey, this is quite an outdated version of the extension, can you update to the current version (0.1.49) and see if the problem persists?

cabljac avatar Apr 22 '24 08:04 cabljac

I am seeing this on the latest version of the extension.

luke-rogers avatar Jun 19 '24 10:06 luke-rogers

Hi @madmacc @luke-rogers ,

Thanks for all the initial details, we are currently looking into this issue.

Thank you for your patience and collaboration! :) Cheers! Gustavo

Gustolandia avatar Dec 16 '24 13:12 Gustolandia

Hi all,

I believe the root of this issue was intermittent errors in Cloud Tasks. I believe it's a duplicate of https://github.com/firebase/extensions/issues/2133

in v0.1.56 we there was a major refactor which would mitigate this issue: Instead of immediately enqueuing a cloud task, we instead attempt several times to directly write to BigQuery. On this (and later) versions of the extension, this issue should basically not be seen.

In theory we could still see a 502 error, but as I understand this will be extremely unlikely, and we haven't seen it reproduced by the extension since 0.1.56.

I am going to close this issue as a duplicate/as resolved. If you see this error occur on version 0.1.56 of the extension, please feel free to reopen and tag me

Thanks! Jacob

cabljac avatar Jan 03 '25 13:01 cabljac