Big Export Failure: Deadline exceeded

[super9xman]

Hello,

I have around 350k documents in 1 of my collections and I am trying to export this data out of firestore...However, I run into the below error. Is there a way to get batch export? I tried --chunk, but it does not work for exports...

Any other solution? Please advice..

Error:

Failure: { Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded at Object.exports.createStatusError (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\common.js:91:15) at ClientReadableStream._emitStatusIfDone (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client.js:233:26) at ClientReadableStream._receiveStatus (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client.js:211:8) at Object.onReceiveStatus (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:1272:15) at InterceptingListener._callNext (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:618:8) at C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:1029:24 code: 4, metadata: Metadata { _internal_repr: {} }, details: 'Deadline Exceeded' } PS C:\Users\JD\Desktop\Learn\practice\firestore-migrator> fire-migrate export --no-subcolls allquotes.json quotes Getting selected collections... Failure: { Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded at Object.exports.createStatusError (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\common.js:91:15) at ClientReadableStream._emitStatusIfDone (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client.js:233:26) at ClientReadableStream._receiveStatus (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client.js:211:8) at Object.onReceiveStatus (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:1272:15) at InterceptingListener._callNext (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus (C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:618:8) at C:\Users\JD\Desktop\Learn\practice\firestore-migrator\node_modules\grpc\src\client_interceptors.js:1029:24 code: 4, metadata: Metadata { _internal_repr: {} }, details: 'Deadline Exceeded' }

[stildalf]

Hi super9xman,

Do you have an idea on which bounds we're hitting here, based on Firestore's quotas? Number of records, request size, timeout, etc.

You're right, fire-migrate only batches uploads/writes for the moment. I'll see if I can put a PR together for --chunk on exports.