Marvin Zhang

**Describe the bug** MongoDB network issue results in Crawlab master node shutdown. Master container logs are as below. ``` crawlab_example_master | server selection error: server selection timeout, current topology: {...

Support directory structure like this. ```bash items - item1.py - item2.py - item3.py middlewares - middleware1.py - middlewares2.py spiders - spider1.py - spider2.py ``` _Originally posted by @TalentedBastard in https://github.com/crawlab-team/crawlab/issues/1047#issuecomment-1004675456_

【优化】scrapy项目上传，爬虫解压的逻辑优化

1

scrapy项目上传，爬虫解压的逻辑 正常在本地打包项目，如果进入项目中进行压缩感觉有些反人类的操作 在后台创建文件的目录树： ```shell └── test2 ├── md5.txt ├── scrapy.cfg └── test2 ├── __init__.py ├── __pycache__ │   ├── __init__.cpython-38.pyc │   └── settings.cpython-38.pyc ├── items.py ├── middlewares.py ├── pipelines.py ├── settings.py...

爬虫详情的scrapy配置，能不能支持读取多种配置文件。我看了下源码，目前只读取了default配置。default一般为本地配置，放服务器上面老是解析报错。 _Originally posted by @selerisjava in https://github.com/crawlab-team/crawlab/issues/776#issuecomment-651491865_

建议添加日志下载功能 _Originally posted by @LBatsoft in https://github.com/crawlab-team/crawlab/issues/776#issuecomment-689927324_

目前ArtiPub还不支持图片上传的功能，需要加入图床来支持图片上传

现在的Cookie状态校验逻辑是判断首页是否包含“登录”二字 ( https://github.com/crawlab-team/artipub/blob/master/spiders/base.js#L341 )，但不少网站的首页内容（例如文章标题、摘要）已经包含“登录”，会导致校验失败

Golang support

4

**Is your feature request related to a problem? Please describe.** It seems emerge does not support the popular programming language Golang **Describe the solution you'd like** Support golang