Zerorigin issues

Results 17 issues of


                                            Zerorigin

【功能增强】消息通知优化

消息通知不支持长任务错误警告，仅支持短任务结束后推送错误提醒。希望能支持长任务实时错误日志通知，方便及时迭代相应项目的业务代码。另外遇到 “cannot find spider files, please re-upload” 问题时，消息通知也未推送。另外消息通知暂不支持加签的 WebHook 机器人通知，希望后期能完善下加签的支持。目前仅能在机器人设置使用关键词或IP段来做安全限制，比较不灵活。

enhancement

logging

project

notification

增加“情景计划任务”功能

情景计划任务： 1、监控 MQ 的特定键集，队列不为空且相应的爬虫未运行，则按照配置好的参数启动对应的爬虫； 2、MQ 队列为空且超过指定时长，则关闭对应爬虫。该想法充分利用分布式爬虫依赖 MQ 队列来存储及消费的特点，实现及时启动或关闭特定爬虫以释放其占用的计算资源。另外想法 2 可以不用理会，scrapy 自带的扩展就能用来实现自动关闭爬虫自身，其它非 scrapy 爬虫也能用少量代码实现在空闲时自动关闭自身。

enhancement

scrapy

完善主从分布式爬虫开发时的业务编排示例文档

看流程图，有提到支持主从分布式数据采集但未在官方文档看到相关章节的业务代码示例

Can't select more than 8 fields from clickhouse due to Tuple.

Can't select more than 8 fields from clickhouse due to `Tuple`. https://docs.microsoft.com/en-us/dotnet/api/system.valuetuple-8?view=net-6.0 ``` SELECT f1, f2, f3, f4, f5, f6, f7, f8, f9 FROM db.table WHERE ID = 'UUID' LIMIT...

希望能動態屏蔽比特彗星 BitComet

### qBittorrent version and Operating System qBittorrent Enhanced Edition 4.1.6 x64 on Windows 10（10.0.18362.53 / X64） ### What is the problem 希望能動態屏蔽比特彗星 BitComet ### What is the expected behavior...

How to remove listening port (5443) and protocol scheme (http/https) from the cache_key when it is not regular port while preserving the hostname?

``` # Caddyfile config block { admin off order cache before rewrite cache { mode bypass allowed_http_verbs GET HEAD stale 1h key { disable_body disable_method disable_query } } } (spec_cache)...

Impossible to open the Nuts DB.db.buildIndexes error: crc error

``` # error detail # caddy validate Caddyfile 2024/04/20 09:20:20.648 INFO using adjacent Caddyfile 2024/04/20 09:20:20.648 INFO using provided configuration {"config_file": "Caddyfile", "config_adapter": ""} 2024/04/20 09:20:21.245 ERROR http.handlers.cache Impossible to...

Zerorigin