aws-sdk-perl icon indicating copy to clipboard operation
aws-sdk-perl copied to clipboard

#221 uri encoding fixes

Open castaway opened this issue 6 years ago • 6 comments

This mostly involves ensuring we use the newly released version of URI::Template, which means we do not double-encode escaped characters. Also added - URI encoding calls die() if it encounters any of AWS's "avoid" characters.

castaway avatar Jul 18 '18 14:07 castaway

Before merging this for release: I understand that upgrading to 0.39 (with this fix applied) will make objects uploaded to S3 with older versions not findable (since they were double escaped, and now arent).

I'm thinking we should put a very explicit notice in the Changelog so that people know of this situation, and a bit of guidance of what scenarios will be of problems to them.

pplu avatar Aug 02 '18 11:08 pplu

Very good point.

It will be possible to find them, but only by using the explicit current name of the object. Eg old name: Fred Bloggs - stored as Fred%20Bloggs to find using new release, lookup using: Fred%20Bloggs

How about something like:

BREAKING CHANGE: Versions of Paws pre 0.39 would incorrectly store object names URI encoded, eg " " is stored as "%20". To retrieve these objects using version 0.39 and above, explicitly ask for the URI encoded name.

?

Might be best to add a line in the POD as well.

castaway avatar Aug 02 '18 11:08 castaway

Hmm.. testing this.. hang on - new code is not yet in release/0.39, right? - oops, I need to downgrade URI::Template for test..

castaway avatar Aug 02 '18 11:08 castaway

Ah I'm wrong - we can't fetch the pre-0.39 objects, as now "%" is disallowed as a Key character, and the code dies().

Would it be worth adding a workaround, maybe an environment variable that could be set, to turn off that dying? I've just tested and it seems to work fine, fetching "Fred%20Bloggs" if we remove the code for disallowing % in keys..

castaway avatar Aug 02 '18 12:08 castaway

S3 was declared unstable, so I wouldn't go through great lengths to re-support broken behavior. If someone depends on it, they should "hack" their way back into the broken state (delete the die?, downgrade URI::Template?). We can inform appropiately as to what has changed, so they can be aware of the situation.

pplu avatar Aug 03 '18 07:08 pplu

@shadowcat-mst wanted to have a think about it. It occurs to me that people can also use the AWS web front-end to rename the broken ones, I think. At the moment I'm thinking something like:

BREAKING CHANGE:Versions of Paws pre 0.39 would incorrectly store object names URI encoded, eg " " is stored as "%20". These will not be retrievable using Paws 0.39+.

castaway avatar Aug 06 '18 09:08 castaway