AIL-framework Configuration documentation?

Hello,

I gave a try to AIL with docker. It starts correctly. after I filled some terms in Terms Management but I don't see anywhere where I can configure where AIL should fetch data, which websites (pastebin...) and at which frequency.

Where are those options? Didn't find any official documentation (No wiki, AIL-framework/doc/ seems more about code/architecture)

Thanks

Oct 12 '16 02:10 juju4

@juju4 This is separated from AIL.

If you want to some raw data via ZMQ, you can get in touch directly with us.

Alternatively you can setup pystemon with the Redis connector:

https://github.com/CIRCL/pystemon/blob/master/pystemon.yaml

and use the feeder built-in in AIL:

https://github.com/CIRCL/AIL-framework/blob/master/bin/feeder/pystemon-feeder.py

I hope this helps.

Oct 12 '16 07:10 adulau

Thanks @adulau ! will review that!

Oct 13 '16 02:10 juju4

Hello, @adulau! You said one can get in touch directly with you for raw ZMQ data. What should i do for data and instructions on how to use it? :)

Nov 09 '16 17:11 AverageS

@AverageS is this still of interest?

Jun 26 '17 17:06 SteveClement

Hi,

Pystemon does not work (will file an issue on the project's repo), there is this page https://github.com/CIRCL/AIL-framework/blob/04b9d9fc1d65d1758200e2a9ac1a52db9ee88569/HOWTO.md which has information about crawlers but it has all been striken out. When I "send to spider" a crawl task, I get an error: ERROR: Script: http://127.0.0.1:8052 SPASH DOWN

Jul 13 '20 02:07 ag-michael