micropython-lib

micropython-lib copied to clipboard

This change adds the following methods from the Response class of the Python requests module to the urequests Response class:

• iter_lines()
• iter_content()
• __enter__()
• __exit__()
• __iter__()

This enables two things:

• processing of large HTTP responses
• succinct handling of the need to close Responses.

So for example:

# Iterate over individual lines of the response
with urequests.get('http://jsonplaceholder.typicode.com/users') as response:
    for line in response.iter_lines():
        print(line.decode(response.encoding))
		
# Iterate over 'chunks' of the response
with urequests.get('http://jsonplaceholder.typicode.com/users') as response:
    for chunk in response.iter_content():
        print(chunk.decode(response.encoding))

The __iter__() method of Response allows this example:

for chunk in requests.get('http://jsonplaceholder.typicode.com/users'):
    print(chunk.decode('UTF-8'))

I chose to reduce the default size of ITER_CHUNK_SIZE from 512 to 128 to conserve resources, not sure this makes sense?

I have tested this new functionality on ESP8266 and ESP32.

Finally, this obviously increases code size, so maybe the scope can be reduced? The __iter__() method could be dropped as this would not reduce the actual capability. Beyond that we could maybe drop the iter_lines() method? Both __enter__() and __exit__() are small and get rid of nasty try finally syntax so I think these should stay?

This feature looks very helpful. I'm having memory errors with urequests on an ESP32. Are there plans to merge?

@chrisb2 just a thank you for the code here, it not only works but it does sort of blow one's mind that in 2021 micropython doesn't bake this in.

Looks great! @chrisb2 would you consider publishing this under a different name? There are at least three variations of the urequests package living in pip already (https://pypi.org/search/?q=urequests) Possibly: chrisb2-urequests