firebase-functions
firebase-functions copied to clipboard
firebase
Getting a protobuf decode error
Related issues
Perhaps https://github.com/googleapis/google-cloud-node/issues/5000
[REQUIRED] Version info
node: 18
firebase-functions: 5.0.1
firebase-tools: 13.11.4
firebase-admin: 12.2.0
[REQUIRED] Test case
Seems someone did it here https://github.com/motorro/BeforeSnapshot
[REQUIRED] Steps to reproduce
Create a onDocumentUpdated Firestore trigger
[REQUIRED] Expected behavior
The function runs
[REQUIRED] Actual behavior
I get this error
Error: Failed to decode protobuf and create a before snapshot.
at entryFromArgs (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/logger/index.js:130:19)
at Object.error (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/logger/index.js:116:11)
at createBeforeSnapshotFromProtobuf (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/common/providers/firestore.js:77:16)
at createBeforeSnapshot (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/v2/providers/firestore.js:187:65)
at makeChangedFirestoreEvent (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/v2/providers/firestore.js:235:37)
at func (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/v2/providers/firestore.js:315:32)
at /workspace/node_modules/.pnpm/@[email protected]/node_modules/@google-cloud/functions-framework/build/src/function_wrappers.js:119:25
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
From the log output
{
"textPayload": "Error: Failed to decode protobuf and create a before snapshot.\n at entryFromArgs (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/logger/index.js:130:19)\n at Object.error (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/logger/index.js:116:11)\n at createBeforeSnapshotFromProtobuf (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/common/providers/firestore.js:77:16)\n at createBeforeSnapshot (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/v2/providers/firestore.js:187:65)\n at makeChangedFirestoreEvent (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/v2/providers/firestore.js:235:37)\n at func (/workspace/node_modules/.pnpm/[email protected][email protected][email protected]_/node_modules/firebase-functions/lib/v2/providers/firestore.js:315:32)\n at /workspace/node_modules/.pnpm/@[email protected]/node_modules/@google-cloud/functions-framework/build/src/function_wrappers.js:119:25\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
"insertId": "667c3eb500038c9a65c10541",
"resource": {
"type": "cloud_run_revision",
"labels": {
"project_id": "gouach-staging-9d1fc",
"service_name": "desiredstateupdatetrigger",
"configuration_name": "desiredstateupdatetrigger",
"revision_name": "desiredstateupdatetrigger-00061-man",
"location": "europe-west1"
}
},
"timestamp": "2024-06-26T16:15:49.232602Z",
"severity": "ERROR",
"labels": {
"instanceId": "005b70877056e0512852391aac057be89361874f0a604a076177c42e1eb85145f92149994b7f0465a809d5e648cf3e05b5e612929d73df6ccd22b2b8167045a3e1",
"goog-managed-by": "cloudfunctions",
"execution_id": "w1erjkpayvvv"
},
"logName": "projects/gouach-staging-9d1fc/logs/run.googleapis.com%2Fstderr",
"receiveTimestamp": "2024-06-26T16:15:49.403958432Z",
"errorGroups": [
{
"id": "CPTJ2fj3s5D8zwE"
}
]
}```
### Were you able to successfully deploy your functions?
yes
I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
Hey. We took a closer look at your repo, and it seems you have a dependency on @google-cloud/tasks. Could you try updating that library to >=5.2.1? It seems to be fixed in that version and above from the issue you linked on google-cloud-node
Hey @exaby73 so to be clear, the above linked repo is from someone else, but I have some code which doesn't require (at least not as a direct dependency) google-cloud/tasks, but still have the same issue when using Firebase functions
Hey @maelp. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.
If you have more information that will help us get to the bottom of this, just add a comment!
Since there haven't been any recent updates here, I am going to close this issue.
@maelp if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.
Sorry for not getting back to you on this for a while. Could you provide a sample which reproduces this?
Unfortunately I don't have a sample right now, I'm trying to update all my project dependencies to latest version and I'll let you know if this fixes something
Okay please do! This is most probably an issue upstream and you may have a peer dependency on cloud tasks from another dependency. You could check your package-lock.json file to check if that's the case
I confirm I still have the issue with the latest packages, here are the packages I'm importing:
"dependencies": {
"@eslint/eslintrc": "3.1.0",
"@eslint/js": "9.5.0",
"@google-cloud/firestore": "7.9.0",
"@google-cloud/functions-framework": "3.4.1",
"@gouach/backend-utils": "workspace:*",
"@gouach/db-schemas": "workspace:*",
"@ts-rest/core": "3.45.2",
"@types/duplexify": "3.6.4",
"algoliasearch": "4.23.3",
"async-mqtt": "2.6.3",
"async-sema": "3.1.1",
"axios": "1.7.2",
"axios-retry": "4.4.1",
"cookie-parser": "1.4.6",
"cors": "2.8.5",
"date-fns": "3.6.0",
"date-format-parse": "0.2.7",
"discord.js": "14.15.3",
"dotenv": "16.4.5",
"eslint": "9.5.0",
"express": "4.19.2",
"firebase-admin": "12.2.0",
"firebase-functions": "5.0.1",
"globals": "15.6.0",
"protobufjs": "7.3.2",
"ramda": "0.30.1",
"vite-tsconfig-paths": "4.3.2",
"zod": "3.23.8"
},
"devDependencies": {
"@google-cloud/logging": "11.1.0",
"@types/cookie-parser": "1.4.7",
"@types/cors": "2.8.17",
"@types/express": "4.17.21",
"@types/node": "20.14.8",
"@types/ramda": "0.30.0",
"@types/ws": "8.5.10",
"@typescript-eslint/eslint-plugin": "7.13.1",
"@typescript-eslint/parser": "7.13.1",
"commander": "12.1.0",
"eslint": "9.5.0",
"eslint-config-google": "0.14.0",
"eslint-config-prettier": "9.1.0",
"eslint-plugin-import": "2.29.1",
"eslint-plugin-prettier": "5.1.3",
"firebase-tools": "13.12.0",
"form-data": "4.0.0",
"isolate-package": "1.18.0",
"open": "10.1.0",
"prettier": "3.3.2",
"tsup": "8.1.0",
"tsx": "4.15.7",
"typescript": "5.5.2",
"vitest": "1.6.0",
"yaml": "2.4.5"
},
And here's the output of cat pnpm-lock.yaml|grep google | pbcopy
'@google-cloud/error-reporting':
google-auth-library:
google-spreadsheet:
version: 4.1.2([email protected]([email protected]))
googleapis:
'@google-cloud/firestore':
'@google-cloud/functions-framework':
'@google-cloud/logging':
eslint-config-google:
'@google-cloud/firestore':
eslint-config-google:
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@google-cloud/[email protected]':
'@googleapis/[email protected]':
'@types/[email protected]':
[email protected]:
[email protected]:
[email protected]:
[email protected]:
[email protected]:
[email protected]:
google-auth-library: ^8.8.0 || ^9.0.0
google-auth-library:
[email protected]:
[email protected]:
[email protected]:
'@google-cloud/[email protected]([email protected])':
'@googleapis/sqladmin': 18.0.0([email protected])
google-auth-library: 9.11.0([email protected])
'@google-cloud/[email protected]([email protected])':
'@google-cloud/projectify': 3.0.0
'@google-cloud/promisify': 3.0.1
google-auth-library: 8.9.0([email protected])
'@google-cloud/[email protected]([email protected])':
'@google-cloud/projectify': 4.0.0
'@google-cloud/promisify': 4.0.0
google-auth-library: 9.11.0([email protected])
'@google-cloud/[email protected]([email protected])':
'@google-cloud/common': 4.0.3([email protected])
'@google-cloud/[email protected]([email protected])':
google-gax: 4.3.7([email protected])
'@google-cloud/[email protected]([email protected])':
google-gax: 4.3.7([email protected])
'@google-cloud/[email protected]':
'@google-cloud/[email protected]([email protected])':
'@google-cloud/common': 5.0.1([email protected])
'@google-cloud/paginator': 5.0.2
'@google-cloud/projectify': 4.0.0
'@google-cloud/promisify': 4.0.0
google-auth-library: 9.11.0([email protected])
google-gax: 4.3.7([email protected])
'@google-cloud/[email protected]':
'@google-cloud/[email protected]': {}
'@google-cloud/[email protected]': {}
'@google-cloud/[email protected]': {}
'@google-cloud/[email protected]': {}
'@google-cloud/[email protected]': {}
'@google-cloud/[email protected]([email protected])':
'@google-cloud/paginator': 5.0.2
'@google-cloud/precise-date': 4.0.0
'@google-cloud/projectify': 4.0.0
'@google-cloud/promisify': 4.0.0
google-auth-library: 9.11.0([email protected])
google-gax: 4.3.7([email protected])
'@google-cloud/[email protected]([email protected])':
'@google-cloud/paginator': 5.0.2
'@google-cloud/projectify': 4.0.0
'@google-cloud/promisify': 4.0.0
google-auth-library: 9.11.0([email protected])
'@googleapis/[email protected]([email protected])':
googleapis-common: 7.2.0([email protected])
'@types/[email protected]': {}
[email protected]([email protected]):
'@google-cloud/firestore': 7.8.0([email protected])
'@google-cloud/storage': 7.11.2([email protected])
'@google-cloud/cloud-sql-connector': 1.3.2([email protected])
'@google-cloud/pubsub': 4.5.0([email protected])
google-auth-library: 9.11.0([email protected])
'@google-cloud/cloud-sql-connector': 1.3.2([email protected])
'@google-cloud/pubsub': 4.5.0([email protected])
google-auth-library: 9.11.0([email protected])
'@google-cloud/cloud-sql-connector': 1.3.2([email protected])
'@google-cloud/pubsub': 4.5.0([email protected])
google-auth-library: 9.11.0([email protected])
[email protected]([email protected]):
[email protected]([email protected]):
[email protected]([email protected]):
google-auth-library: 9.11.0([email protected])
[email protected]:
[email protected]([email protected]([email protected])):
google-auth-library: 9.11.0([email protected])
[email protected]([email protected]):
google-auth-library: 9.11.0([email protected])
[email protected]([email protected]):
google-auth-library: 9.11.0([email protected])
googleapis-common: 7.2.0([email protected])
google-p12-pem: 4.0.1
'@types/google.maps': 3.54.10
'@types/google.maps': 3.54.10
workbox-google-analytics: 7.1.0
[email protected]:
I guess I'm running just off-the-shelf cloud functions, they are defined liked this:
export const desiredStateUpdateTrigger = onDocumentUpdated(
{
document: `${dbCollectionNames.batteryBMS}/{id}`,
vpcConnector: isProduction ? "api-pingo-connector" : undefined,
vpcConnectorEgressSettings: isProduction
? "PRIVATE_RANGES_ONLY"
: undefined,
},
async (event) => {
const snap = event.data;
if (snap === undefined) {
return;
}
const oldData = snap.before.data();
const newData = snap.after.data();
const id = event.params.id;
....process data
}
and it's when doing this that I have the error
Could you possibly provide a sample of the data in one of these documents, with any sensitive data redacted of course so that I may try to reproduce it?
I don't think it's linked to any particular data, because this happens in ANY of my "onDocumentUpdated" methods, which have very different data and schemas
I think the protobuf encoding/decoding error must be at the level of the Firebase event serialization/deserialization, not related to the actual data in the Firebase (eg it's probably related to the serialization of the "event" message by the Google Cloud Functions lib)
This is my tsconfig in case
{
"compilerOptions": {
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "Bundler",
"composite": true,
"noImplicitReturns": true,
"noUnusedLocals": true,
"allowJs": true,
"strict": true,
"skipLibCheck": true,
"types": ["node"],
"lib": [
"ESNext",
"ESNext.AsyncIterable",
],
"baseUrl": ".",
"typeRoots": [
"./node_modules/@types"
],
"paths": {
"@/*": ["./src/*"],
"@backend-utils": ["./node_modules/@gouach/backend-utils/src/index.ts"],
"@backend-utils/*": ["./node_modules/@gouach/backend-utils/src/*"],
"@schemas": ["./node_modules/@gouach/db-schemas/src/index.ts"],
"@schemas/*": ["./node_modules/@gouach/db-schemas/src/*"]
},
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"resolveJsonModule": true
},
"compileOnSave": true,
"include": [
"src", "scripts", "lib-scripts"
]
}
and I'm using the "isolate" package to move local monorepo pnpm dependencies in the local "cloud functions" dir when deploying on Firebase (otherwise it doesn't find local dependencies)
https://www.npmjs.com/package/isolate-package
Here's my isolate config
{
"buildDirName": "./dist",
"includeDevDependencies": false,
"isolateDirName": "isolate",
"logLevel": "info",
"tsconfigPath": "./tsconfig.json",
"workspaceRoot": "../",
"excludeLockfile": false,
"avoidPnpmPack": false
}
(also I don't know why this error doesn't seem to trigger an "error report email" from GCloud which I normally have when my cloud function fail)
@exaby73 so I investigated some more, and it doesn't seem to be an import version of a lib or whatever, but rather the import order.
My functions is an index.ts which exports many other files, eg
export * from "someFileA.ts";
export * from "someFileB.ts";
...
and I could solve the issue just by moving one of those file at the bottom, eg
export * from "someFileA.ts";
export * from "someFileB.ts";
...
///////////
// add an empty line just to make sure the order will be preserved
export * from "someFileWhichCausedIssues.ts";
absolutely no idea why
- the "someFileWhichCausedIssues" was located because if I removed it from the imports originally, the other cloud functions would start working, but if I put it back, and removed almost everything, just adding a tiny "onDocumentUpdated" method it would fail
- although if I added the same onDocumentUpdated-triggered cloud-function in the index.ts after all the "export *" it would work
- so this lead me to try and import the file at the bottom, and this worked
I have no idea why
Is it possible for you to find out which imports from any Firebase or Google Cloud pakcages causes this and in which order?
Ran into the same issue today. Below is how I've split my code.
src/activity-calendar/activity-calendar-trigger.ts
import { logger } from 'firebase-functions/v2';
import { onDocumentWritten } from 'firebase-functions/v2/firestore';
import { updateRemoteConfigs } from '../remote-config';
import constants from '../constants';
export const activityCalendarTriggerImpl = () => {
return onDocumentWritten('activity-calendar/{country}/detail/{docId}', async event => {
try {
....
} catch (error) {
logger.log('onWriteActivityCalendar trigger error :', error);
}
});
};
src/index.ts
import { setGlobalOptions } from 'firebase-functions/v2';
import * as admin from 'firebase-admin';
const serviceAccount = require('../serviceAccountKey.json');
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
storageBucket: constants.FIREBASE_CONSTANTS.STORAGE_BUCKET,
});
import { activityCalendarTriggerImpl } from './activity-calendar/activity-calendar-trigger';
const DEFAULT_LOCATION = constants.FIREBASE_CONSTANTS.DEFAULT_FUNCTIONS_REGION;
setGlobalOptions({ region: DEFAULT_LOCATION });
const onWriteActivityCalendar = activityCalendarTriggerImpl();
export {
onWriteActivityCalendar
}
This gets deployed with no issue but when the trigger is invoked it gives the same issue
I changed the gen 2 function to gen 1 in activity-calendar/activity-calendar-trigger.ts
This caused no issue.
import * as functions from 'firebase-functions';
import { updateRemoteConfigs } from '../remote-config';
import constants from '../constants';
const DEFAULT_LOCATION = constants.FIREBASE_CONSTANTS.DEFAULT_FUNCTIONS_REGION;
export const activityCalendarTriggerImpl = () => {
return functions
.region(DEFAULT_LOCATION)
.firestore.document('activity-calendar/{country}/detail/{docId}')
.onWrite(async (change, context) => {
try {
...
} catch (error) {
functions.logger.log('onWriteActivityCalendar trigger error :', error);
}
});
};
After testing out further found that this is happening for all the triggers I've written using gen 2
@PasinduDG Could you check your dependencies as well as mentioned above in the thread?
@exaby73
Here are the packages that I'm importing.
"dependencies": {
"@google-cloud/tasks": "^4.0.1",
"axios": "^1.5.1",
"dotenv": "^16.3.1",
"firebase-admin": "^12.2.0",
"firebase-functions": "^5.0.1",
"jsonwebtoken": "^9.0.0",
"jwk-to-pem": "^2.0.5",
"moment": "^2.29.4",
"moment-timezone": "^0.5.43",
"pdfmake": "^0.2.7",
"sharp": "^0.32.6"
},
"devDependencies": {
"@types/jest": "^29.5.12",
"@types/node": "^17.0.41",
"@typescript-eslint/eslint-plugin": "^7.0.2",
"@typescript-eslint/parser": "^7.0.2",
"eslint": "^8.17.0",
"eslint-config-google": "^0.14.0",
"eslint-config-prettier": "^9.1.0",
"eslint-config-standard": "^17.1.0",
"eslint-plugin-import": "^2.29.1",
"eslint-plugin-n": "^16.6.2",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-prettier": "^5.1.3",
"eslint-plugin-promise": "^6.1.1",
"firebase-functions-test": "^3.3.0",
"jest": "^29.7.0",
"poeditor-cli": "^1.1.1",
"prettier": "^3.2.5",
"ts-jest": "^29.1.5",
"typescript": "^4.8.0"
}
Also, the output of cat yarn.lock | grep google | clip.exe
"@google-cloud/firestore@^7.7.0":
resolved "https://registry.yarnpkg.com/@google-cloud/firestore/-/firestore-7.9.0.tgz#fcacef51c07e39c083bad3a4801814a7a60a878a"
google-gax "^4.3.3"
"@google-cloud/paginator@^5.0.0":
resolved "https://registry.yarnpkg.com/@google-cloud/paginator/-/paginator-5.0.2.tgz#86ad773266ce9f3b82955a8f75e22cd012ccc889"
"@google-cloud/projectify@^4.0.0":
resolved "https://registry.yarnpkg.com/@google-cloud/projectify/-/projectify-4.0.0.tgz#d600e0433daf51b88c1fa95ac7f02e38e80a07be"
"@google-cloud/promisify@^4.0.0":
resolved "https://registry.yarnpkg.com/@google-cloud/promisify/-/promisify-4.0.0.tgz#a906e533ebdd0f754dca2509933334ce58b8c8b1"
"@google-cloud/storage@^7.7.0":
resolved "https://registry.yarnpkg.com/@google-cloud/storage/-/storage-7.11.2.tgz#a8942d9a4a75634186d262b82c5b41e90649f11c"
"@google-cloud/paginator" "^5.0.0"
"@google-cloud/projectify" "^4.0.0"
"@google-cloud/promisify" "^4.0.0"
google-auth-library "^9.6.3"
"@google-cloud/tasks@^4.0.1":
resolved "https://registry.yarnpkg.com/@google-cloud/tasks/-/tasks-4.1.0.tgz#c5905bc76cb0d949cb7b45bff18f155927e52210"
google-gax "^4.0.4"
eslint-config-google@^0.14.0:
resolved "https://registry.yarnpkg.com/eslint-config-google/-/eslint-config-google-0.14.0.tgz#4f5f8759ba6e11b424294a219dbfa18c508bcc1a"
"@google-cloud/firestore" "^7.7.0"
"@google-cloud/storage" "^7.7.0"
google-auth-library@^9.0.0:
resolved "https://registry.yarnpkg.com/google-auth-library/-/google-auth-library-9.4.1.tgz#dea32cbdae0a47066995a379e6873c52926ea80e"
google-auth-library@^9.3.0, google-auth-library@^9.6.3:
resolved "https://registry.yarnpkg.com/google-auth-library/-/google-auth-library-9.11.0.tgz#bd6da364bcde4e0cc4ed70a0e0df5112b6a671dd"
google-gax@^4.0.4:
resolved "https://registry.yarnpkg.com/google-gax/-/google-gax-4.0.5.tgz#e30a2f2fad3716e8b23996f1bbe941e16abf0889"
google-auth-library "^9.0.0"
google-gax@^4.3.3:
resolved "https://registry.yarnpkg.com/google-gax/-/google-gax-4.3.7.tgz#f1870902d09c54c5d1735ef1ee7903d4458d6a49"
google-auth-library "^9.3.0"
I see you're using an older version of Google Tasks. Could you follow the steps highlighted in https://github.com/firebase/firebase-functions/issues/1579#issuecomment-2200589340 and confirm if that fixed it or not?
@exaby73 this started working for me after upgrading @google-cloud/tasks to latest v5.5.0
Hey @maelp. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.
If you have more information that will help us get to the bottom of this, just add a comment!
Since there haven't been any recent updates here, I am going to close this issue.
@maelp if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.