Akshay Sharma comments

Results 18 comments of


                                            Akshay Sharma

Wrong type(response) for binary responses

Hello @Gallaecio, I have submitted a proposal for this year's GSoC program. Here is the link to it: https://docs.google.com/document/d/1X9g62mNxYI305nfiAAmYkrOWkQMiaToni7kIWx30TKA/edit?usp=sharing My apologies for showing it to you this late, I wasn't...

Integrating xtractmime into Scrapy

What I understand by looking into https://github.com/scrapy/scrapy/blob/master/scrapy/responsetypes.py, I think `from_args` is the main function required by other scrapy files for mime sniffing. I think calling `xtractmime.extract_mime` with different parameters based...

Integrating xtractmime into Scrapy

> Related to that, although not achievable simply extending `CLASSES`: the standard taught me that any MIME type ending in `+xml` is to be treated as an XML file, so...

Integrating xtractmime into Scrapy

What can be the value of the `supported_types` parameter for `extract_mime`? Is that required here or not?

Integrating xtractmime into Scrapy

I have added the pre n post xtractmime tests with expected behavior as comments. There can be more failing scenarios, if I found one I will add it later. Still,...

Integrating xtractmime into Scrapy

> E AssertionError: {'headers': {b'Content-Disposition': [b'attachment; filename="data.xml.gz"']}, 'url': 'http://www.example.com/page/'} ==> != This is failing because `mimetypes.MimeTypes()` returning a `text/xml` content type instead of a `application/gzip` ``` >>> MimeTypes().guess_type("data.xml.gz") ('text/xml', 'gzip')...

Integrating xtractmime into Scrapy

> E AssertionError: {'body': b'\x00\xfe\xff', 'url': 'http://www.example.com/item/', 'headers': {b'Content-Type': [b'text/plain']}} ==> != This is failing as we are not considering NULL byte anymore and xtractmime detecting `b"\xfe\xff"` as a `text/plain`...