Madison Bahmer

Not a web interface for generating spider, but a web interface for visualizing the API data generated from working with the cluster. The REST services wrapper that interacts with Kafka...

Apache NiFi is nice but is not the UI I am looking for. Picture a very plain ui like for Hadoop, Hbase, Spark, Storm or another popular open source Apache...

Pushing this back to 1.3, with the focus on getting a solid rest service (#24) for 1.2

The rest service does not take care of that, it sounds like you would need a customized spider to do better duplicate detection, or to modify the very vanilla RFPDupefilter...

@damienkilgannon Thanks for your interest! I have a bit more work to do on the Rest passthrough endpoint (Kibana logs) but it is pretty much ready to go. I built...

Those look like good first mock ups, you certainly have got my wheels turning on what we would really like to see in a UI! Keep in mind we can...

Mockup looks great! I finally was able to tidy up the rest component so now that is merged into the `dev` branch and dockerized - so it should make for...

@villeristi both #174 and #116 are in a partial working state to get the UI into the main branches.

@deed02392 Does the python 3 containers not work for you right now? The `master` branch is not up to date, but the project on the `dev` branch is fully python...

All of the unit and integration tests for the python3 docker container continue to pass, here is the build from lasst week. https://travis-ci.org/github/istresearch/scrapy-cluster/jobs/664497952 I'm not sure there is much more...