centipede-crawler
centipede-crawler copied to clipboard
Session authentification
Related to umpirsky/centipede#8.
- Add session auth + allow custom implementation
- Add ignore URLs (compatible with yml file umpirsky/centipede#2)
Now looks like :
public function __construct($baseUrl, $depth = 1, $ignoreUrl = array(), AuthenticatorInterface $authenticator = null)
What do you think ?
Hi @ProPheT777, thanks for your PR.
I would like to support list of ignore urls on centipede config file level, not on crawler level.
Regarding authentication, need to think a bit...
Yeah authentication need more reflection but it's a first approach, currently I take the more simple way to achieve it.
I'm working to move exlude urls logic directly on the callback via yml
:+1: