Downloads stall with `connResetException` error

[yoda-vid]

I have checked that the SDK documentation doesn't solve my issue.

• [X] Yes I've checked them.

I have checked that the API documentation doesn't solve my issue.

• [X] Yes I've checked them.

I have checked that the Box Developer Forums doesn't solve my issue.

• [X] Yes I've checked them.

I have searched Issues in this repo and my issue isn't already reported.

• [X] Yes I've checked them.

Describe the bug

I have run into periodic stalls when downloading folders. Typically 100-200 files will be downloaded successfully, followed by a connResetException error in earlier Box CLI versions and more recently simply hanging. I have tried every x.y Box CLI version from 3.0 to 3.12 with Node.js 16.20.2 (also 20.11.0 for Box 3.12.2). I am running it on AWS in an Ubuntu 20.04 EC2 instance and have tried both m6a and m6in instances in case it might be a networking throughput issue. I'm on a Box Enterprise account.

Expected behavior

Download all files in the folder without error or stall, or at least infrequent errors.

Steps to reproduce

Here is the output when I run this command: box folders:download <id> --destination=/data -v

  box:@box/cli:hooks:init done +0ms
  box init version: @oclif/[email protected] argv: [ 'folders:download', 'xxx', '--destination=/data', '-v' ] +0ms
  box-cli:init Initializing Box CLI +0ms
  box-cli:init Loaded settings {
  box-cli:init   boxReportsFolderPath: '/home/ubuntu/Documents/Box-Reports',
  box-cli:init   boxReportsFileFormat: 'txt',
  box-cli:init   boxDownloadsFolderPath: '/home/ubuntu/Downloads/Box-Downloads',
  box-cli:init   outputJson: false,
  box-cli:init   enableProxy: false,
  box-cli:init   proxy: { url: null, username: null, password: null },
  box-cli:init   numMaxRetries: 3,
  box-cli:init   retryIntervalMS: 3000,
  box-cli:init   uploadRequestsTimeoutMS: 90000
  box-cli:init } +1ms
  box-cli:init Using environment xxx {
  box-cli:init   clientId: ‘xxx’,
  box-cli:init   clientSecret: ‘xxx’,
  box-cli:init   name: ‘xxx’,
  box-cli:init   cacheTokens: true,
  box-cli:init   authMethod: 'oauth20'
  box-cli:init } +0ms
  box-cli:init SDK client settings {
  iterators: true,
  analyticsClient: [Object],
  request: [Object],
  numMaxRetries: 3,
  retryIntervalMS: 3000
} +1ms
  box-cli:execute Starting execution command: folders:download argv: [ 'xxx', '--destination=/data', '-v' ] +0ms
  box-cli:execute Running framework error handler +45m
  box-cli:execute Handling re-thrown error in base command handler +38ms
Error: aborted
    at connResetException (node:internal/errors:705:14)
    at TLSSocket.socketCloseListener (node:_http_client:454:19)
    at TLSSocket.emit (node:events:525:35)
    at node:net:301:12
    at TCP.done (node:_tls_wrap:588:7)

The error occurs after downloading about 100-200 files, each file 40-100 MB.

Starting in Box CLI 3.7.0, I no longer received the error message, but the download just stalls.

I have tried with and without the timeout configuration as suggested in https://github.com/box/boxcli/issues/414#issuecomment-1265075553 with similar results.

Could this be an issue with Box on AWS?

Authentication method used in your application

User Authentication (OAuth 2.0)

App Access Level

App + Enterprise Access

What is Box CLI Version and Node used?

Box CLI 3.12.2 (also tried 3.0.0 through this version), Node 16.20.2 (also tried 20.11.0)

What is your Operating System Version?

Ubuntu 20.04

[mhagmajer]

Hello @yoda-vid, thanks for posting the issue!

Debugging folder download scenarios is difficult because the process is dependant on the contents of the folder you're downloading as well as the networking and platform conditions. Let us investigate this further by following a few cycles where we add more debugging information to get to the root cause of the issue you're facing.

First of all, when we release a new version, will you be able to upgrade it on your side and post the verbose download output here until the process hangs/throws errors? Also, do these problems happen consistently on every try in your environment?

[yoda-vid]

Thank you, @mhagmajer! I completely understand that debugging this is challenging and appreciate your help.

Yes, I can upgrade and post verbose update, and the issue has been occurring consistently with each try. I recently built a binary with the --no-overwrite (#516), which has improved the process somewhat by allowing me to rerun the download with re-downloading all the files already downloaded. I couldn't find specific Linux binary build instructions but followed this Gist, except replacing Node v10 with v16. Each run has also worked for longer (eg 500ish files), which I imagine is related to networking/platform conditions.

Please let me know if I should be outputting more verbose output than above. With the latest version, the CLI just hangs with the download terminal graphic still spinning and no error message. On a slightly different note regarding verbose output, after I added the overwrite flag, the Skipping file ... at ... because it already exists and overwrite is disabled message appears for all skipped files over and over. The only workaround I found to filter these lines was to run the command through script -c "<cmd>" /dev/null | grep -wv "^Skipping".

[lukaszsocha2]

Hi @yoda-vid, we have just release a new CLI v13.3.0 with improved logging. Can you please upgrade it on your side and try to reproduce the error? The output as above is good enough. Best, @lukaszsocha2

[yoda-vid]

Thank you, @lukaszsocha2! I will try it out and keep you posted.

[yoda-vid]

I have been using CLI v13.3.0 and (alas, for better or for worse) networking connectivity has been been much better with no error or stall encountered thus far. I will continue to test and keep you posted. Thank you for your help with this troubleshooting.

[stale[bot]]

This issue has been automatically marked as stale because it has not been updated in the last 30 days. It will be closed if no further activity occurs within the next 7 days. Feel free to reach out or mention Box SDK team member for further help and resources if they are needed.

[stale[bot]]

This issue has been automatically closed due to maximum period of being stale. Thank you for your contribution to Box CLI and feel free to open another PR/issue at any time.