dmjio
Implement signature version 4
I just received the following error when trying the URL from s3-signer:
<Error>
<Code>InvalidRequest</Code>
<Message>
The authorization mechanism you have provided is not supported.
Please use AWS4-HMAC-SHA256.
</Message>
<RequestId>...</RequestId>
<HostId>...</HostId>
</Error>
Is this related to not using Signature Version 4?
Amazon S3 supports Signature Version 4, a protocol for authenticating inbound API requests to AWS services, in all AWS regions. At this time, AWS regions created before January 30, 2014 will continue to support the previous protocol, Signature Version 2. Any new regions after January 30, 2014 will support only Signature Version 4 and therefore all requests to those regions must be made with Signature Version 4.
Yea, when I created this library the regions used were compatible. It seems they are deprecated now.
A new AWS library that exports v4 signing capabilities has come along.
http://hackage.haskell.org/package/aws-general-0.2.2/docs/Aws-SignatureV4.html
I believe its just a matter of migrating to this library.
@spl I can take look at implementing this on Thursday 2/23. How soon do you need this by?
Thanks for the pointer to aws-general. I hadn't yet looked into other libraries. Looks like there's also aws-sign4. It doesn't seem like hS3 supports v4. It's also not yet clear to me whether these support overriding headers.
I'm currently evaluating options and plan to implement something this week. If you're going to look into v4 and #4 this week, I'm happy to work with you. Otherwise, if you suggest using another library, that's fine, too.
I was just trying to use aws-general, but I'm not sure if I'm either (1) doing it right, or (2) it is not working, or both.
First, it's missing the eu-central-1 region, which I added (alephcloud/hs-aws-general#7).
Second, I'm not sure what the request payload is in canonicalRequest. Is it "/<bucket>/object"?
Third, it calls for an “Action parameter.” I see that in the v4 documentation, but I don't see it in the S3 documentation.
I'm now looking at amazonka. I may just go with that since I'll need other things besides URL signing.
@spl hm, that's interesting, seems like there's some discrepancies here. I might refresh this package anyways, since it appears I'm going to need something internally to issue upload URLs for .mp4 files.
heh, sounds good :)
On Wed, Feb 24, 2016 at 12:43 PM, Sean Leather [email protected] wrote:
Cool. I'll watch and see what you come up with. :)
— Reply to this email directly or view it on GitHub https://github.com/dmjio/s3-signer/issues/1#issuecomment-188400302.
Cell: 1.630.740.8204
oh crap I spent a bunch of time integrating this into my app and then I saw this issue :(
which is a shame because this api was great/easy to use. aws-general doesn't look as nice...
@jb55, ah yea, sorry about that. Although, it might be possible for us to call into aws-general, using the S3Keys and S3Request types from this package.
@dmjio There's some extra cruft in aws-general. I'm working on porting just the signing stuff into this package
So it's basically working if anyone wants to give it a quick look-over/spin:
https://github.com/jb55/s3-signer/compare/ab718c34208895ff75d1f6fea2b51aef2ec87be0...jb55:sigv4
from cli patch-by-patch:
git fetch https://github.com/jb55/s3-signer sigv4 && git log --reverse -p ab718c34208895ff75d1f6fea2b51aef2ec87be0..FETCH_HEAD
or all at once:
git fetch https://github.com/jb55/s3-signer sigv4 && git diff ab718c34208895ff75d1f6fea2b51aef2ec87be0..FETCH_HEAD
I have a quick manual test in test/Test.hs
I've tested it and it generates the correct signature from their examples. I'm going to test it in my client application and then work on documentation updates afterwards.
Cheers,
Something I thought about this morning. I've been reading the docs on the deprecation status of V2, because I was wondering why it was still working for some of my old applications. Apparently it's still supported in older regions but not in new regions new going forward.
My current patchset removes V2 signing, but perhaps we should keep it with its current api, and then add a new V4 module namespace for the new stuff. That way the library won't break for existing applications, but the new method will be available there.
The API is significantly different that it might not make sense making a different "backend" for generateS3URL, I believe it would be cleaner if it was a completely separate API.
Let me know what yall think.
@jb55 Fantastic. You're a hero for saving this package. Regarding v3 support, I'm fine making your changes backwards incompatible with others. What we can do is make a v3 branch, and then push your changes into master on 4.0 (cabal version). All users that rely on the old version (v3) can apply the proper upper bound on their packages. If a bug is detected in v3 and it needs fixing, that is fine, we can patch it in the v3 branch, bump the 3.x major version and then upload it to hackage. It will coexist harmoniously with your 4.x versions.
@jb55 I've gone ahead an made a v3 branch for users that still rely on this older version. If you make a PR with your changes I would love to review and accept as the official v4.
@jb55 do you think the proposed versioning scheme will accommodate the needs of your applications on both v3 and v4 schemes?
@dmjio It doesn't affect me personally, it's possible it might be annoying for people migrating to the new version in the same application. The probability of that might be low so it may not be a problem? Up to you. Bumping the major version is definitely simpler.
Sounds good, then let's proceed with major bump plan. PR when ready and I'll merge, we can add tests potentially before hackage release.