Paweł Miech

What do you think about Selector(response).xpath().map() ?

3

Sometimes I'd like to apply some function after extracting something, and I do something like this: ```python In[32]: map(json.loads, sel.xpath("//@data-p13n-asin-metadata").extract()) Out[32]: [{u'asin': u'B00Y2863GQ', u'price': 221.99, u'ref': u'pd_bxgy_75_1_1'}, {u'asin': u'B008J3UD2U', u'price':...

Allow spider attr

5

This is about https://github.com/scrapinghub/scrapyjs/issues/14 and https://github.com/scrapinghub/scrapyjs/issues/11 opening this for discussion, needs tests and improvements, would be cool to get some feedback if it's in good direction

There is not much about deploying scrapyrt, there is only one short section in wiki. There should be more info in readthedocs.

add support for request kwargs in GET handler

1

looking [here](https://github.com/scrapinghub/scrapyrt/blob/master/scrapyrt/resources.py#L110) there is comment saying: ``` At the moment kwargs for scrapy request are not supported in GET. They are supported in POST handler. ``` this is inconvenient, we...

it will default to logging.DEBUG but users can set this to something higher in case they dont want all the noise. fixes #30 for future we can also add command...

We should add some command line argument to allow to customize logging, this would be useful for running scrapyrt in production but also good for apps that inherit from scrapyrt...

[docs] add missing async def to sensors/monitors section

1

## Description sample code won't work without async def. I spotted this when reading docs here: https://faust.readthedocs.io/en/latest/userguide/sensors.html#monitor

add tutorial showing how to build something useful with Twisted Klein

8

[Flask](http://flask.pocoo.org/docs/tutorial/) or [Django](https://docs.djangoproject.com/en/1.10/intro/tutorial01/) have nice tutorials that guide users through building some simple web application. It should be rather simple to write something like this for Twisted Klein. It should...

http proxy support

2

there is currently no way to use treq with HTTP proxies, this is main reason I had to use twistd.web.client.ProxyAgent in some recent project. What do you think about adding...