No way to export more than 2000 items

[daniberv]

Throws different errors when exporting composite on Windows. There is definitely some issues with so many and fast requests about gcs-resumable-upload.json. So I can't export the collection because the process stucks on 2100-2200 items every time. npm cache clear doesn't work too.

I tried to add some interval in exportItem method but still no luck. In this case it throw another errrors:

<...path...>\toolbox\functions\node_modules\@google-cloud\storage\build\src\storage.js:92
>                  const reason = (_a = e.reason) === null || _a === void 0 ? void 0 : _a.toLowerCase();
>                                         ^
> 
>  TypeError: Cannot read property 'reason' of null
>      at Object.RETRYABLE_ERR_FN_DEFAULT [as retryableErrorFn] (<...path...>\toolbox\functions\node_modules\@google-cloud\storage\build\src\storage.js:92:40)
>      at Pumpify.<anonymous> (<...path...>\toolbox\functions\node_modules\@google-cloud\storage\build\src\file.js:2927:51)
>      at Pumpify.emit (events.js:315:20)
>      at Pumpify.Duplexify._destroy (<...path...>\toolbox\functions\node_modules\duplexify\index.js:195:15)
>      at <...path...>\toolbox\functions\node_modules\duplexify\index.js:185:10
>      at processTicksAndRejections (internal/process/task_queues.js:79:11)
BadRequestError: request aborted
    at IncomingMessage.onAborted (C:\Users\User\.cache\firebase\tools\lib\node_modules\raw-body\index.js:231:10)
    at IncomingMessage.emit (events.js:315:20)
    at IncomingMessage.EventEmitter.emit (domain.js:482:12)
    at abortIncoming (_http_server.js:532:9)
    at socketOnClose (_http_server.js:525:3)
    at Socket.emit (events.js:327:22)
    at Socket.EventEmitter.emit (domain.js:482:12)
    at TCP.<anonymous> (net.js:674:12)
BadRequestError: request aborted
    at IncomingMessage.onAborted (C:\Users\User\.cache\firebase\tools\lib\node_modules\raw-body\index.js:231:10)
    at IncomingMessage.emit (events.js:315:20)
    at IncomingMessage.EventEmitter.emit (domain.js:482:12)
    at abortIncoming (_http_server.js:532:9)
    at socketOnClose (_http_server.js:525:3)
    at Socket.emit (events.js:327:22)
    at Socket.EventEmitter.emit (domain.js:482:12)
    at TCP.<anonymous> (net.js:674:12)
BadRequestError: request aborted
    at IncomingMessage.onAborted (C:\Users\User\.cache\firebase\tools\lib\node_modules\raw-body\index.js:231:10)
    at IncomingMessage.emit (events.js:315:20)
    at IncomingMessage.EventEmitter.emit (domain.js:482:12)
    at abortIncoming (_http_server.js:532:9)
    at socketOnClose (_http_server.js:525:3)
    at Socket.emit (events.js:327:22)
    at Socket.EventEmitter.emit (domain.js:482:12)
    at TCP.<anonymous> (net.js:674:12)
BadRequestError: request aborted
    at IncomingMessage.onAborted (C:\Users\User\.cache\firebase\tools\lib\node_modules\raw-body\index.js:231:10)
    at IncomingMessage.emit (events.js:315:20)
    at IncomingMessage.EventEmitter.emit (domain.js:482:12)
    at abortIncoming (_http_server.js:532:9)
    at socketOnClose (_http_server.js:525:3)
    at Socket.emit (events.js:327:22)
    at Socket.EventEmitter.emit (domain.js:482:12)
    at TCP.<anonymous> (net.js:674:12)
>  
>  events.js:292
>        throw er; // Unhandled 'error' event
>        ^
>  FetchError: request to http://localhost:9199/upload/storage/v1/b/test-nft-888d1.appspot.com/o?uploadType=multipart&name=%2Fcandy-machine-8b8b9087-43b0-482d-8b1a-b3c1dba47da6 failed, reason: socket hang up
>      at ClientRequest.<anonymous> (<...path...>\toolbox\functions\node_modules\node-fetch\lib\index.js:1461:11)    
>      at ClientRequest.emit (events.js:327:22)
>      at Socket.socketOnEnd (_http_client.js:453:9)
>      at Socket.emit (events.js:327:22)
>      at endReadableNT (_stream_readable.js:1221:12)
>      at processTicksAndRejections (internal/process/task_queues.js:84:21)
>  Emitted 'error' event on Pumpify instance at:
>      at errorOrDestroy (<...path...>\toolbox\functions\node_modules\readable-stream\lib\internal\streams\destroy.js:98:101)
>      at Pumpify.onerror (<...path...>\toolbox\functions\node_modules\readable-stream\lib\_stream_readable.js:704:47)
>      at Pumpify.emit (events.js:315:20)
>      at Pumpify.Duplexify._destroy (<...path...>\toolbox\functions\node_modules\duplexify\index.js:195:15)
>      at <...path...>\toolbox\functions\node_modules\duplexify\index.js:185:10
>      at processTicksAndRejections (internal/process/task_queues.js:79:11) {
>    type: 'system',
>    errno: 'ECONNRESET',
>    code: 'ECONNRESET'
>  }

[skeletoncrewrip]

hi! the newer versions of the code dont make a new network call on every exportItem. Are you up to date with the repo?

[daniberv]

Hi! Yes I'm definitely up to date. It throws something like permitted...gcs-resumable-upload.3162563631.json rename gcs-resumable-upload.json

[martinwhitman]

Just delete that file or rename it to 'gcs-resumable-OLD' or whatever, and try again.

[daniberv]

I've tried it. No luck

[Mathieuduverney]

Any fix? I have the same issue

[skeletoncrewrip]

New updates were made on Nov 29. Please make sure you are running the latest version of the code, and have re-built inside the functions directory.

[Mathieuduverney]

I did ..

[Mathieuduverney]

Still have this issue with latest version and clean set up If someone find any solution would be helpful