Support more advanced features in plugin-spider-assitant

[tikazyq]

Support directory structure like this.

items
- item1.py
- item2.py
- item3.py

middlewares
- middleware1.py
- middlewares2.py

spiders
- spider1.py
- spider2.py

Originally posted by @TalentedBastard in https://github.com/crawlab-team/crawlab/issues/1047#issuecomment-1004675456