Close CallbackFileWrapper.__buf once it's used to free memory.

[elnuno]

This tries to solve #145. It seems like CallbackFileWrapper.__buf is keeping its stream after we're done with it, and it unnecessarily bloats the process.

Running the script at the end (and attached) against a 74 MB file I get...

For original code:

Using 106 MB on program end. Mean memory use: 199 MB

For both patched code and bare requests:

Using 32 MB on program end. Mean memory use: 32 MB

(These numbers are on Windows, Python 3.4.2 32-bits)

import os
import logging
import shutil

import requests
import cachecontrol
import psutil

# logging.basicConfig(level=logging.DEBUG)

us = psutil.Process(os.getpid())
MB = 1024 * 1024
N = 15

sess = cachecontrol.CacheControl(requests.Session())

total = 0
for i in range(N):
    url = 'http://localhost:8000/bigdata.bin?limit=%s' % i
    print('Requesting %s...' % i)
    response = sess.get(url, stream=True)
    fh = open('dest.bin', 'wb')
    shutil.copyfileobj(response.raw, fh)
    fh.close()
    used_mem = us.memory_full_info().uss / MB
    total += used_mem
    print('Using %d MB.' % round(used_mem))

import gc
gc.collect()

print('Done.')
print('Using %d MB on program end.' % round(us.memory_full_info().uss / MB))
print('Mean memory use: %d MB' % round(total/N))
input()

[ionrock]

I realize this is pretty old, but do you mind rebasing on the latest master?

[itamarst]

This was merged in #254 and can be closed.