PseudoKV migration acceleration

[MeijeSibbel]

##Motivation Faster migration optimizes contract formation as contracts can be re-used to storage data faster. Furthermore if migration is faster less contracts will have to be formed which is advantageous both in terms of financial and in terms of scalability.

Current behavior:

The velocity of data migration < new data uploaded --> New contract set will have to be formed to continue uploading.

Currently migration is slow as can be seen on the snippet below from one of our gateways:

2020-10-22T12:03:14.494Z        DEBUG   grpc/client.go:104      pulled a storage class  {"path": "/ddos1-2/file.ezpp", "request_id": "e86c8e14d667e0f0903f2fb08b4750f2591f2fdc", "class": {"gatewayID":"a74982c8-67a1-4dae-a366-f082c85bc5d0:67e1a599-f654-4bb0-af52-2958768811dd","userID":"a74982c8-67a1-4dae-a366-f082c85bc5d0","bucket":"ddos1-2","className":"STANDARD"}, "active_contract": "6f38b62b-5017-4f1d-9097-7229819e5bea", "reserved_contract": "0528065b-7a62-4d80-8feb-c5fde1875c65", "migrating_contracts": ["6d7d6d2e-c789-40ad-98cd-7b14ece377cb", "c5689dfd-f047-41dc-8c3c-c81aaf871119", "8878808b-1b2a-40ac-a5c6-5ff8a6bf391c", "f1111543-3ce6-4646-a171-7d38da78233c", "a1ef4748-7fe6-4821-a3b7-dba0dfb5fe68", "07ba7094-6119-4caa-8f54-dc9e4a5fe175", "d9e0b28d-bb6a-4b8b-8c43-92ea63d6a9d9", "e608807d-9d07-4639-a1ed-cd7370ace921", "1c91412b-c9df-440a-a857-b3ef2c3805ec", "2a65461e-f8e7-4669-a7e6-3904ddcdaf40", "7b7c4a68-0b83-4cb9-be43-6977373a0db6", "33c25ce3-aa50-455b-9809-3d0eb9997d86", "602815dd-804e-4ba6-840a-e2b5f567e692", "169b6e19-c352-4c42-8329-e84ca9d3b87c", "b3770c9b-9a96-4596-b4eb-306bc5d156e8", "8ba48fc3-f13c-4444-a769-2d9804eb1b29", "25bc62e1-72f0-46fa-b0dc-2fb767655116", "164edbc8-656c-4d15-bb24-6a5e8902d4bd", "4aecceee-329a-4d6e-83f7-64876d6eec75"], "waiting_contracts": [], "full_contracts": ["5e64aea0-fa89-418a-9a44-24c3b099c0ed", "5c585ef8-ab51-4f25-85bf-6c2041d62c78", "4d6821c2-e57b-4284-b7b8-05cfc5a2687f", "578bd561-0b99-47f7-a518-01a10c27b5a8", "a44abcbb-df3e-4f57-b27e-b6ab87a7c5b5"]}

Many contract sets are stuck in the migration state (19 contract sets ~ 19*40 = 760 contracts).

If migration is carried out from a resource deficient or low upload throughput host, migration can take a very long time to complete. The impact of this grows as more data is stored.

Expected behavior

Migrate from the fastest host(s) with the highest upload throughput to minimize migration time. As overdrive will be optimizing for slow hosts during upload, it would be beneficial to have something similar for migration.