spidermon

spidermon copied to clipboard

Create a CLI for setup a very basic spidermon monitors and notifications

17

Some simple prompt questions that help new comers to setup a basic spidermon installation in a easy and quick way. ``` $ spidermon setup $ To who should I send...

raphapassini

Action to restart spider in Scrapy Cloud

4

Discussed in chat a bit - the idea is that if a job meets some conditions (monitor detects certain website responses, the job stalls, etc) this action could restart the...

andrewbaxter

Include support for Cerberus as optional item validation library

2

Actually Spidermon allows the user to choose between jsonschema or schematics as item validation library. We would like to include [cerberus](https://github.com/pyeve/cerberus) as a new choice to the user.

rennerocha

Include documents about monitor mixins

3

We have different mixins in `spidermon/contrib/monitors/mixins` directory, but no documentation.

rennerocha

No dep on slack-sdk

1

In #338 slackclient was replaced by slack-sdk in setup.py only in `test_requirements` but not in `extras_require`, so installing `spidermon[monitoring]` doesn't install it and the Slack action no longer works.

wRAR

Adding use of ItemAdapter to prevent assumptions of item nature

3

https://github.com/scrapinghub/spidermon/issues/353

further-reading

Change format of content of _validation field (#425)

7

The content of _validation field must to be the string representation of a Python dict instead defaultdict when SPIDERMON_VALIDATION_ADD_ERRORS_TO_ITEMS setting is True.

rochamatcomp

Change format of content of _validation field

1

If a validation error is found and [SPIDERMON_VALIDATION_ADD_ERRORS_TO_ITEMS](https://spidermon.readthedocs.io/en/1.20.0/item-validation.html#spidermon-validation-add-errors-to-items) setting is `True`, we populate the `_validation` field with the errors. The content of this field is the string representation of a...

rennerocha

`SPIDERMON_MONITOR_SKIPPING_RULES` (https://github.com/scrapinghub/spidermon/pull/384) doesn't appear to be mentioned in the docs nor the release notes, even though it appears to be available in 1.20.0 (https://github.com/scrapinghub/spidermon/blob/release-1.20.0/spidermon/contrib/scrapy/monitors/base.py#L29C18-L29C50). The setting is documented in the...

curita

Default email template fails with latest `scrapinghub` client version

1

## Issue `data.job.metadata` (an instance of `scrapinghub.client.jobs:JobMeta`) cannot be subscribed. There are multiple places in Spidermon's templates where job meta keys are accessed via `data.job.metadata[x]`. Those cases fail because of...

curita