This plugin is not using the tornado runloop and thus is very slow!

[RobertBiehl]

UPDATE Ah, I missed the pull request in async S3 HTTP with botornado #14 Please merge! :)

ORIGINAL POST When comparing the loader and storage classes in this plugin with the standard http loader for example (https://github.com/thumbor/thumbor/blob/master/thumbor/loaders/http_loader.py) then you'll see that thumbor_aws is blocking the whole runloop when fetching files from s3. This is a bad practice when working with tornado and defeats the purpose of thumbor, which usually is very fast. A possible solution would be to use tornado-botocore (https://github.com/nanvel/tornado-botocore) instead, which allows htto requests through the standard tornado.httpclient.

E.g. compare https://github.com/thumbor/thumbor/blob/master/thumbor/loaders/http_loader.py#L87 client.fetch(req, callback=partial(return_contents, url=url, callback=callback, context=context)) (proper callback usage) with https://github.com/willtrking/thumbor_aws/blob/master/thumbor_aws/loaders/s3_loader.py#L58 return callback(file_key.read()) (blocks the runloop)

[vvh-empora]

+1

[Luisehahne]

+1

[cmuskalla]

+1

[burnerx]

+1

[janlem]

+1

[DarkViper]

+1

[SebastianBoldt]

+1

[ndc-empora]

+1

[RobertBiehl]

Ah, I missed the pull request in async S3 HTTP with botornado #14 Please merge! :)

[BaconFist]

+1

[aectann]

Doesn't seem to be a problem anymore since Thumbor 5.0 got released. Need to fix the code to support 5.0 though. Here is a PR for that: https://github.com/willtrking/thumbor_aws/pull/24