Optimise MAV FTP upload to improve performance

[rmackay9]

We have a report in Copter-4.4 that uploading a 30MB terrain file using MAVFTP takes >30min (see report here)

As part of the investigation I attempted to upload a 30MB terrain file (produced using https://terrain.ardupilot.org/) and confirmed that indeed MP does take at least 30min (it only uploaded 1MB of the file in 5min). See MP issue here: https://github.com/ArduPilot/MissionPlanner/issues/3229

To try and determine if MAVProxy was any faster (in the hopes that it would be thus confirming that the issue was in MP) I found that MAVProxy's "ftp" command simply doesn't work at least on windows.

I was running the latest MAVProxy (1.8.66)

The exact procedure was:

• started MAVProxy and connected to a CubeOrange running Copter-4.4.2 (stable) or Copter-4.5.0-dev (latest)
• copied the N43E143.DAT file to my Desktop
• attempted to ftp to the APM directory (I had also tried uploading to APM/TERRAIN earlier with no success)
• waiting >7min and checked the since of the file, it was still zero

STABILIZE> ftp put C:\Users\rmack\Desktop\N43E143.DAT APM/
Putting C:\Users\rmack\Desktop\N43E143.DAT as APM/N43E143.DAT
STABILIZE> ftp list APM
Listing APM
 D LOGS
 D STRG_BAK
 D TERRAIN
 D scripts
   N43E143.DAT  0
   UNKNOWN.bak  16384

By the way, if I ran the same ftp command but neglected to add the "/" after the "APM" I got this message spamming my console.

[stephendade]

I'm able to upload using a similar setup (Copter on master, Pixhawk1, MAVProxy from install 1.8.66). It is quite slow - around 15-25kByte/sec.

A 33Mb terrain file took ~33min - so 1Mb/min transfer rate.

If you use ftp status during the transfer, is it showing progress?

[rmackay9]

I've changed the title to reflect that this is no longer considered a bug but rather an enhancement request.

On the Nov 14th dev call we discussed with Tridge and discovered that MAVFTP's implementation (both within the GCSs and the flight code) has been optimised for downloading (e.g. from the autopilot->GCS) but not for uploading (e.g. GCS->autopilot).

There were two suggestions for improving peformance:

1. send multiple MAVFTP messages from the GCS to the autopilot without waiting for a response.
2. create a new MAVLink message to send longer messages just like we do for downloading. Currently it seems the MAVFTP messages are only 80bytes long

I will still try @stephendade's suggestion to confirm that the FTP is working on windows even if it is very slow.

[rmackay9]

@stephendade (and anyone else who is interested),

I ran, "ftp status" during the transfer and saw what's below so it is making (slow) progress.