Elendil
- As the SDK is generated from api version 2017-08-01-preview which is configured from https://github.com/Azure/azure-rest-api-specs/tree/main/specification/security/resource-manager#tag-package-composite-v3 And some of the properties we see when using Azure CLI of api version 2020-01-01-preview...
We need to do some check to see if user's spider is written correct
Scheduler part in current design is not elegent, re-write it
Both Downloader and Spider node should register themself in Consul, and expose webapi so that an web admin page can use these web api to know how many Downloader and...
NScrapy Downloader/Spider node can be wrapped to a WepAPI to support service discovery of Micro-Service architecture, currently the 2 nodes are hardly to be wrapped into an WebAPI
Considering to refactor the structure of the project---big change