Simulate multipart/form-data file upload with Falcon's Testing module

[dirkschneemann]

This simple Falcon API will take a HTTP POST with enctype=multipart/form-data and a file upload in the file parameter, returning the file's content to the caller:

# simple_api.py
import cgi
import falcon
import json

class MultipartFileUpload(object):
    def on_post(self, req, resp):
        upload = cgi.FieldStorage(fp=req.stream, environ=req.env)
        data = upload['file'].file.read().decode('utf-8')
        resp.body = json.dumps(dict(data=data))

app = falcon.API()
app.add_route('/', MultipartFileUpload())

To try this out, create a textfile with some content and send to the API via curl -F "[email protected]" localhost:8000.

I would like to test this API now with Falcon's testing utilities, simulating such a file upload. The simulate_request method has a file_wrapper parameter which might be useful but from the documentation I do not understand how it is supposed to be used or whether this is possible at all.

As a workaround, I currently use webtest where this is supported, another possibility would be werkzeug. Anyway, I'm happy to drop dependencies if this should be possible with Falcon itself.

[JamesRamm]

@dirkschneemann

Here is how I test posting a zip archive to one of our endpoints. The test function:

def test_posted_model_gets_saved(self, client):
        '''The API can accept a data archive, create and
            return a ``Model`` document pointing to the
        location of the extracted folder
        '''
        path = create_zip()
        with open(path, 'rb') as zfile:
            fake_zip_bytes = zfile.read()
        response = client.simulate_post(
            '/models',
            body=fake_zip_bytes,
            headers={'content-type': 'application/zip'}
        )
        assert response.status == falcon.HTTP_CREATED
        assert 'Location' in response.headers
        assert '_id' in response.json

the POST handler:

def on_post(self, req, resp):
         # Save and extract the archive
        directory, name = self._archive_store.save(req.stream, req.content_type)

        # Create the mongodb metadata document
        model = self._document(
            name=name,
            folder=directory
        ).save()

        # build response
        resp.status = falcon.HTTP_CREATED
        resp.location = '/models/{}'.format(model.name)
        resp.body = model.to_json()

As you can see, it actually uploads a file, rather than mocking it. For using file_wrapper; Line 182 of test_hello.py in the falcon tests shows an example of using it. At the top of the file is the FileWrapper class.

My testing strategy here is broadly similar to the 'functional tests' part of the tutorial: http://falcon.readthedocs.io/en/stable/user/tutorial.html#functional-tests

[amirma]

This is what I came up with based on what my Chrome does when I upload a file.

[vytas7]

This is an interesting proposal. I'll drop a link to my in-progress issue so I don't forget to revise it later: https://github.com/falconry/falcon/issues/953 As I was crafting lots of test cases for that work-in-progress anyway, we could well generalize that as a testing client feature.

FWIW, in addition to the tools mentioned by @dirkschneemann and @amirma's Gist, yet another option for the time being is using requests_toolbelt.MultipartEncoder to construct the payload. That is obviously an external dependency though.

To summarize, I'll try to address this myself, but probably only after we're done with Falcon 3.0. So I'm leaving the needs contributor label for now, feel free to volunteer!

[vytas7]

@samajain as I understand you're working on this? Pls confirm.

[samajain]

Hi @vytas7,

Can I get started in a few days? I am interested in working on this but I have my finals going on right now. I will start the work from this weekend and share an update. Thanks!

[vytas7]

Sure @samajain! This has been dormant for a couple of years, it is totally fine if you are starting next week :sweat_smile:

[samajain]

Hey @vytas7, I have started working on this. I am going through the documentation on Falcon and trying to understand the framework. Is there any information you would like to share which would be important while working on this task?

[vytas7]

Hi again @samajain! Not sure if there's anything specific just to this issue... :slightly_smiling_face:

Maybe for starters you could write a Falcon app that deals with an uploaded multipart form, then try to write a couple of tests for it constructing a multipart form manually, probably something not too dissimilar to test_media_multipart.py. Once you feel comfortable with all the details, you can "productify" this by introducing a new parameter in the testing interface, something like form or files, or a naming suggestion of yours.

Not sure about the interface of that parameter either, but maybe we could loosely follow the popular requests?

[vytas7]

Hi again @samajain, just checking if you have done any work on this issue?

[vytas7]

In the absence of response, this issue is looking for new contributor(s)! :wave:

[TigreModerata]

I've started looking at this, if anyone else joins in give a shout out and we can coordinate :)