Parallel downloads

[cmandesign]

Behaviour

Steps to reproduce this issue

1. Make config file with proper credentials
2. Start FTP Grabber
3. FTP Grabber starts to download, but single file simultaneously

Expected behaviour

I expected to see multiple downloads when I set connection per host.

Actual behaviour

It only downloads files sequentially, one by one.

Configuration

• FTPGrab version : 5.1.1
• Platform : windows
• Remote FTP : Filezilla-Server-windows

Logs

FTPGrab logs (set --log-level flag to debug and --log-ftp)

[2mThu, 28 Mar 2019 10:09:00 UTC[0m [32mINF[0m Starting FTPGrab 5.1.1
[2mThu, 28 Mar 2019 10:09:00 UTC[0m [33mDBG[0m {
  "Flags": {
    "Cfgfile": ".\\config_niko.yml",
    "Schedule": "",
    "Timezone": "UTC",
    "LogLevel": "debug",
    "LogJson": false,
    "LogFile": "",
    "Docker": false
  },
  "App": {
    "ID": "ftpgrab",
    "Name": "FTPGrab",
    "Desc": "Grab your files periodically from a remote FTP or SFTP server easily",
    "URL": "https://ftpgrab.github.io",
    "Author": "CrazyMax",
    "Version": "5.1.1"
  },
  "Server": {
    "Type": "ftp",
    "FTP": {
      "Host": "95.216.93.225",
      "Port": 21,
      "Username": "********",
      "Password": "********",
      "Sources": [
        "/Apex"
      ],
      "ConnectionsPerHost": 10,
      "Timeout": 5,
      "DisableEPSV": false,
      "TLS": {
        "Enable": false,
        "Implicit": true,
        "InsecureSkipVerify": false
      },
      "LogTrace": false
    },
    "SFTP": {
      "Host": "",
      "Port": 22,
      "Username": "********",
      "Password": "********",
      "Key": "********",
      "Sources": null,
      "Timeout": 30,
      "MaxPacketSize": 32768
    }
  },
  "Db": {
    "Enable": true,
    "Path": "apex_updater.db"
  },
  "Download": {
    "Output": "/Apex",
    "UID": 0,
    "GID": 0,
    "ChmodFile": 420,
    "ChmodDir": 493,
    "Include": null,
    "Exclude": null,
    "Since": "0001-01-01T00:00:00Z",
    "Retry": 3,
    "HideSkipped": false,
    "CreateBasedir": false
  },
  "Mail": {
    "Enable": false,
    "Host": "localhost",
    "Port": 25,
    "SSL": false,
    "InsecureSkipVerify": false,
    "Username": "********",
    "Password": "********",
    "From": "",
    "To": ""
  },
  "File": {
    "FileAttributes": 32,
    "CreationTime": {
      "LowDateTime": 2846841088,
      "HighDateTime": 30729172
    },
    "LastAccessTime": {
      "LowDateTime": 2846841088,
      "HighDateTime": 30729172
    },
    "LastWriteTime": {
      "LowDateTime": 45055992,
      "HighDateTime": 30729548
    },
    "FileSizeHigh": 0,
    "FileSizeLow": 532,
    "Reserved0": 0
  }
}
[2mThu, 28 Mar 2019 10:09:00 UTC[0m [32mINF[0m ########
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [33mDBG[0m 0 entries found in database
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [32mINF[0m ########
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [32mINF[0m Grabbing from /Apex
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [32mINF[0m --------
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [32mINF[0m Checking /Apex/amd_ags_x64.dll
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [32mINF[0m Already downloaded
[2mThu, 28 Mar 2019 10:09:01 UTC[0m [31mWRN[0m Skipped: Already downloaded
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m --------
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m Checking /Apex/audio/ship/audio.mprj
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m Already downloaded
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [31mWRN[0m Skipped: Already downloaded
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m --------
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m Checking /Apex/audio/ship/general.mbnk
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m Exists but size is different
[2mThu, 28 Mar 2019 10:09:02 UTC[0m [32mINF[0m Downloading file (56.03MB) to /Apex/audio/ship/general.mbnk...
[2mThu, 28 Mar 2019 10:16:32 UTC[0m [32mINF[0m File successfully downloaded!
[2mThu, 28 Mar 2019 10:16:32 UTC[0m [32mINF[0m Time spent: 7 minutes
[2mThu, 28 Mar 2019 10:16:32 UTC[0m [32mINF[0m --------
[2mThu, 28 Mar 2019 10:16:32 UTC[0m [32mINF[0m Checking /Apex/audio/ship/general.mbnk_digest
[2mThu, 28 Mar 2019 10:16:32 UTC[0m [32mINF[0m Never downloaded
[2mThu, 28 Mar 2019 10:16:32 UTC[0m [32mINF[0m Downloading file (1.82MB) to /Apex/audio/ship/general.mbnk_digest...
[2mThu, 28 Mar 2019 10:16:47 UTC[0m [32mINF[0m File successfully downloaded!
[2mThu, 28 Mar 2019 10:16:47 UTC[0m [32mINF[0m Time spent: 14 seconds
[2mThu, 28 Mar 2019 10:16:47 UTC[0m [32mINF[0m --------
[2mThu, 28 Mar 2019 10:16:47 UTC[0m [32mINF[0m Checking /Apex/audio/ship/general_english.mstr
[2mThu, 28 Mar 2019 10:16:47 UTC[0m [32mINF[0m Never downloaded
[2mThu, 28 Mar 2019 10:16:47 UTC[0m [32mINF[0m Downloading file (848.7MB) to /Apex/audio/ship/general_english.mstr...

[crazy-max]

Thank you for the report, I understand better now. The ConnectionsPerHost setting is the maximum number of FTP connections per host. This one is useless and will be removed in the next version. As far as simultaneous downloads are concerned, this is not yet in place but I will take into account your request :wink:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.