spidermon
spidermon copied to clipboard
scrapinghub
Create a CLI for setup a very basic spidermon monitors and notifications
Some simple prompt questions that help new comers to setup a basic spidermon installation in a easy and quick way.
$ spidermon setup
$ To who should I send the notification email: [email protected]
$ Do you want to enable Slack Notifications [Y/n]: y
$ Please provide an Slack API key to send notifications: ab213234_cHJscaJRFVN
Thanks for enabling the amazing Spidermon! You're good to go.
@raphapassini I'm new to the opensource world. Can I pick this issue and provide a pull request?
@GrooChu are you working on this feature? In a positive case, I'll update to in progress, otherwise I'll work on it.
Hi, my name is Geancarlo Murillo. I'm want to contibute with Spidermon project in GSoC 2019, I looked here for ideas. I found the Create a CLI to setup spidermon on projects idea for this year and I want to work with it. There is some issue or use cases to start and can write a proposal ? Thanks ;)
Hi @ejulio , I would like to work on this issue for GSOC2019, can you guide me about how can I start with this issue?
hey @riya-17 and @gcmurillo . @gcmurillo , sorry for the late response.
Both of you, and @leovictorsr had shown interest in working with this issue.
And I talking to @rennerocha on what we can expect here and also with other projects we think would be possible to work with Spidermon. We should get back to you with ideas about this project in the following days.
Thank you for your interest in working with us :smile:
@ejulio Thank you.. Will be waiting for the ideas about this project so that we can understand and start contributing soon :)
In the meantime, @riya-17 , @gcmurillo and @leovictorsr, we invite you all to take a look into Spidermon current issues see if there is something you can contribute. This is a good to get you started with the project and code base.
@ejulio Ok, I will have a go through issues and code base and try contributing to them.
@riya-17 we are reviewing/merging one here https://github.com/scrapinghub/spidermon/pull/127 Feel free to give your thoughts as well
@riya-17 we are reviewing/merging one here #127 Feel free to give your thoughts as well
It is merged and available here
This CLI should be used by anyone that want to configure Spidermon quickly. I suggest the use of click to create the command.
Ask the user which built-in monitors they want to enable (and request the necessary settings):
- https://github.com/scrapinghub/spidermon/blob/master/spidermon/contrib/scrapy/monitors.py
Ask the user which built-in actions they want to enable (and request the necessary settings):
- https://github.com/scrapinghub/spidermon/tree/master/spidermon/contrib/actions
Ask the user if they want item validation (options JSON Schema or schematics).
In the end, a new monitors.py file should be created and project settings.py needs to be updated with the settings. If item validation was chosen, create an empty validators.py file or create a templa JSON Schema.
Hi, @rennerocha, I'm looking for work in that issue in GSoC 2019 program...I think thats the beginning for a more complete CLI, can I work on that as a first step ?
Hi, I am Jaydeep Godara and willing to work on this issue as part of GSoC 2019. So whether I directly work on this issue according to above guidelines or is there some qualification task to do first.
hey @JaydeepGo . Nice that you are interested in working with this issue. First, you need to submit a proposal on how you plan to work with this issue and we'll chose among applications.
Hey folks, I forgot to state my intention to work on the Spidermon CLI idea here for Google Summer of Code 2019, all the while contributing to other issues in Spidermon :hatching_chick:
I have finalized my ideas into a proposal, made out of a lot of research, hours and code. I would love if all the maintainers, mentors, and community members could review the same through the Google Summer of Code dashboard, and propose any and all suggestions, comments, and improvements they see in the proposal :heart: Do checkout the feature map. As @raphapassini and @stav suggested, I will also be opening a PR for the CLI plans, would love to get your feedback on the same. Let's get this done right.
CC: @raphapassini @ejulio @rennerocha @andrewbaxter @manycoding @cathalgarvey
