file_generator are not always closed

[ghost]

Originally reported by: Patrik Dufresne (Bitbucket: patrik_dufresne, GitHub: @ikus060)

---

Reference to #1314.

The commit associated to this bug is missing a condition to also close file served using file_generator(). When an error occurred, the file_generator() is not property close (so is the input file). The method is_closable_iterator() is returning false.

I suggest to add a method close() to the file_generator().

---

• Bitbucket: https://bitbucket.org/cherrypy/cherrypy/issue/1402

[webknjaz]

@ikus060

[ikus060]

I'm bumping this issue again.

When serving FileLike Object, cherrypy is not closing the object. This might happen with filedescription, static file, etc.

Here a snippet to reproduce the problem:

import unittest
from urllib.error import HTTPError
import cherrypy
from cherrypy.lib.static import serve_fileobj
from cherrypy.test.helper import CPWebCase
import urllib


class FileLikeObject():
    def __init__(self) -> None:
        self.count = 0
        self.is_closed = False

    def read(self, size):
        if self.count > (1024 * 1024 * 1024):
            return None
        self.count += size
        return '1' * size

    def close(self):
        self.is_closed = True

    @property
    def closed(self):
        return self.is_closed


class Root():
    fd = None

    @cherrypy.expose
    def index(self):
        return 'OK'

    @cherrypy.expose
    @cherrypy.config(**{"response.stream": True})
    def download(self):
        self.fd = FileLikeObject()
        return self.fd

    @cherrypy.expose
    def isclosed(self):
        return str(bool(self.fd and self.fd.closed))


class TestRoot(CPWebCase):

    interactive = False

    @classmethod
    def setup_server(cls):
        cherrypy.tree.mount(Root())

    def test_fd_close(self):
        self.getPage('/isclosed')
        self.assertBody('False')
        # Query the URL but don'T read all of it's content
        opener = urllib.request.build_opener()
        try:
            f = opener.open("http://%s:%s/download" % (self.HOST, self.PORT))
            f.close()
        except HTTPError:
            pass
        # Expect fileobject to be closed
        self.getPage('/isclosed')
        self.assertBody('True')


def main():
    cherrypy.quickstart(Root())


if __name__ == "__main__":
    main()

[ikus060]

~~I believe the TimeoutMonitor should be responsible to identify the serving that are pending and close them on timeout.~~

[ikus060]

After preliminary verification, adding a "close" function to the file_generator class is fixing the problem.

class _file_generator(object):
    """
    Yield the given input (a file object) in chunks (default 64k).
    """

    def __init__(self, input, chunkSize=65536):
        self.input = input
        self.chunkSize = chunkSize

    def __iter__(self):
        return self

    def __next__(self):
        chunk = self.input.read(self.chunkSize)
        if chunk:
            return chunk
        else:
            if hasattr(self.input, 'close'):
                self.input.close()
            raise StopIteration()
    next = __next__

    def close(self):
        if hasattr(self.input, 'close'):
            self.input.close()