spidermon

spidermon copied to clipboard

`SlackClient.api_call` error responses are not logged

2

@jesuslosada @rennerocha Hi, Spidermon team! Recently I started to get these errors in my job logs: ``` ====================================================================== ERROR: SendSlackMessageSpiderFinished ---------------------------------------------------------------------- Traceback (most recent call last): File "/app/python/lib/python3.6/site-packages/spidermon/core/actions.py", line 37,...

alexander-matsievsky

Unable to find scrapy.cfg file to infer project data dir

1

 i got error message like this when deployed scrapy project to scrapyd, even when scrapy.cfg is included in the egg file I have deployed a scrapy project to scrapyd,...

sulthonzh

Currently, there are monitors for item count validation, checking errors, etc. There is one important monitor that is missing, to monitor whether the data has been exported to FEED_STORAGE or...

HassanQamar07

So I was reading #95 and saw that it might be interesting to improve current default integrations, which I had to do some tweaking on slack's. I was wondering about...

thamenato

Support jsonschema CustomValidator

4

As reported in https://github.com/scrapinghub/spidermon/issues/51#issuecomment-480250235 (I cp message here, since the original is too long) We need to support other non natively jsonifiable types or force specific Python types with jsonschema...

manycoding

Related to #186. Projects deployed in [Scrapy Cloud](https://scrapinghub.com/scrapy-cloud) overrides `STATS_CLASS` setting by default. To make `LocalStorageStatsHistoryCollector` works correctly, it is necessary to [manually change](https://support.scrapinghub.com/support/solutions/articles/22000200670-customizing-scrapy-settings-in-scrapy-cloud) the Spider settings to use Spidermon's...

rennerocha

Include Cerberus as new available library for item validation

1

rennerocha

Document use of custom templates

1

Related to #126 , we need a better documentation to explain how to define and use custom templates for e-mails, reports, etc. [This action](https://github.com/scrapinghub/spidermon/blob/master/spidermon/contrib/actions/templates.py) contains part of the code that...

rennerocha

Can Spidermon monitor on database instead of scraped data

2

Is it solely to monitor data and pattern after scraping, or can we monitor the database condition?

vionemc

Replace autodoc with autoapi

1

https://github.com/rtfd/sphinx-autoapi 1. doesn’t run the code, it just parses the files, thus removes any need from installing the package and solves dependencies overhead. 2. doesn’t require you to execute the...

manycoding