fine-uploader
fine-uploader copied to clipboard
FineUploader
It's not possible to supply sessionToken if request is being signed by backend
Hi,
I have the following configuration
<script type="text/javascript">
$(function() {
$('#fineUploader').fineUploaderS3({
template: 'qq-template',
request: {
endpoint: "https://MY-BUCKET-HERE.s3.amazonaws.com",
},
credentials: {
accessKey: "...",
sessionToken: "...",
expiration: "2015-05-06",
},
signature: {
endpoint: "/my_end_point"
},
uploadSuccess: {
endpoint: "index.html"
},
chunking: {
enabled: true,
mandatory: true,
}
});
});
</script>
The headers that end up being POST-ed to the backend are the following
POST
image/gif
x-amz-acl:private
x-amz-date:Tue, 05 May 2015 15:11:11 GMT
x-amz-meta-qqfilename:600-animated.gif
/MY-BUCKET-HERE/9c061d9f-c743-4831-93cd-d4fd7cc72896.gif?uploads
Since x-amz-security-token is missing here, this never gets authenticated by Amazon. I tried plugging sessionToken in different places, e.g. into request but never managed to get it to work.
Also on a side note, for the example above, the header that gets sent to Amazon is
Authorization:AWS null:signture-goes-here. So it appears that credentials.accessKey does not get picked up because it considers credentials set to be incomplete without secretKey in it (which I don't want to put there, I want the request to be signed by backend). I am not sure if it's a bug or intended behavior.
I managed to get all of this working in one of two ways:
- by supplying all 4 values into
credentials - by having
request.accessKeyset and hacking JS code to setx-amz-security-tokenlike thisheaders["x-amz-security-token"] = 'a string with my token'
Version: 5.2.1
All properties of the credentials option, other than accessKey are only intended to be used by client-side signing workflow. If you'd like to sign requests on your server, do not supply any credentials property other than accessKey in your options, and provide a path to your signature server.
@rnicholus I don't entirely understand how the workflow can work in case of a temporary credentials and if I want to use my own signature server. In this case there is no way to supply sessionToken and JS will never add x-amz-security-token header which will lead to AWS rejecting the request.
You can read more about the client-side upload/signing process on the associated feature page in the documentation. If you do not want to handle signing client-side with short-lived credentials, use the server-side signing workflow.
@rnicholus I don't want to do client-signing, my ultimate goal is to have server-side signing workflow. The problem I am facing is that there is no way to have server-side signing workflow function with temporary credentials because there is no way for me to provide sessionToken.
(also thanks for very quick replies!)
It's not clear why you would like to provide temporary credentials when your credentials are stored safely on your server and never exposed to the client.
If you'd like to change your credentials with the server-side signing workflow, you'll need to re-initialize Fine Uploader with a new access key. We can consider a method to update your access key via the API in the future though.
@rnicholus That's a good question, that's more of a company workflow thing. Devops provide us with a place where we can request temporary credentials when necessary and then we can play around with those. Once we have a working prototyple, devops creates a permanent key+secret, we store those safely and deploy (and at this point findeuploader will work perfectly fine). So for production purposes the server-side signing workflow with temp credentials doesn't make any sense but for dev purposes this use case can have its applications in the development cycle (perhaps for large corporations only; I would imagine for smaller ones, it's easier to keep track of credentials without this complicated system).
@rnicholus if you are provisioning server keys via IAM instance profiles (AWS best practices), you must supply the sessionToken, which is currently not possible with fine-uploader (while using backend signing)
@Jud Not being familiar with this workflow, it sounds like this could be solved simply by allowing the server to return this token as part of its response to Fine Uploader's signature request with the expectation that Fine Uploader will then include this token as a request parameter when communicating with S3. Is this correct?
@rnicholus If there was a way to pass through the x-amz-security-token header from the signature response, and have fine-uploader use it when communicating with S3, that would work.
I think this will be quite complicated to deal with server-side, as your server code will then need to reconstruct the policy document (for older browsers) or the headers string (for newer browsers that utilize the multipart upload API) in order to provide a correct signature, since x-amz-security-token must be accounted for when generating a signature.
...and the complexity of dealing with this server-side once v4 signatures are supported by Fine Uploader is much greater.
Running into this issue now when trying to upload from an EC2 instance that has credentials supplied via IAM instance profiles.
Ideally, should be able to provide the session token at Fine Uploader initialization (much like accessKey) (and maybe even dynamically to account for the case when a token may expire in the middle of an upload).
Since the secret key and access key should change in tandem would it make sense to have the signature endpoint return the accessKey? This would ensure each request was signed using the right secret-/access-key pair (I think).
Since the secret key and access key should change in tandem would it make sense to have the signature endpoint return the accessKey?
Yes, I think that is feasible. I'm less familiar with how this session token affect the signature. If it must be considered when signing the request and included as a header on the request to S3, that makes all of this more complicated. I'll need to look into this further before I start implementing a solution.
I don't know if this will be completed as part of 5.7.0, but I will certainly begin researching and planning changes to the library to address the workflows described here. This has become more of a priority as of late. The need to allow for a constantly changing STS token and access key is something we are looking at for our apps that use Fine Uploader internally as well. Associating an EC2 server with an IAM instance profile that contains a set of temporary credentials for signing the request does seem to be a best practice that is being adopted by others. Fine Uploader could very well be sending requests to a cluster of EC2 instances, each with a different set of temporary credentials. This means each signature request for a single file could require different set credentials for each chunk/request. It seems important for Fine Uploader to support this workflow.
I'm hitting this issue now as well. And it's biggie. Basically fineuploader has worked like a dream during development, running on local and dev boxes with non-temporary credentials. But now that we're migrating to boxes with credentials provided by IAM roles, which are temporary and require the header and sessionToken, posts are no longer accepted by S3.
Has someone identified a workaround for this? Hoping I just haven't tried everything yet. Adding the token value via sessionToken doesn't work, the required header isn't sent as part of the request.
AWS docs discuss using temporary credentials at https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html#RequestWithSTS
Open to any and all suggestions
Just to confirm - the workaround detailed by @eugenebond in the case description does not work for you, @trauts2? There's an open pull request - #1549 - that aimed to address this to some degree, but it needs some attention and refactoring, and I'm not sure if it elegantly supports temporary tokens to the degree that I'd like to see.
I'm not sure I follow the workaround. @eugenebond would you mind sharing the configuration that ended up working? What is the 4th bit you added to the credentials block?
Personally my workaround is to not use the IAM Instance Profile until I have a good workaround. I can get away with that for a while, but it would be great to get the plumbing working with temporary credentials since this is arguably weakening the security to put static credentials on boxes.
@trauts2 I haven't worked on this since spring 2015 and my memories are a bit hazy so apologies if something in my reply is inaccurate. From what I remember, the "4th bit" comment was about supplying all 4 of these http://docs.fineuploader.com/api/options-s3.html#credentials My understanding is that Fineuploader's original intention is to have this flow used in cases when you want frontend to handle signing. You might not want to use this because of security implications.
In my case I wanted to use tmp credentials and backend signing approach. I don't remember finding a good fix for this and since, in my case, I had to work with tmp credentials in dev only, I think I ended up hacking something in fineuploader internals (since I anyway had an approximate idea where things were located after trying to debug it). I haven't kept the files and don't remember the codebase anymore so unfortunately I can't advise you what to change, but even then, you probably don't want to do that for prod anyway, because it was some quick and ugly hack.
I'd very much like to have proper STS support in Fine Uploader, or support for temporary credentials with a server-side signing workflow. Unfortunately, I'm pulled in many different directions these days, and simply responding to support requests and bug reports takes the majority of my time. Also, I'm in the process of (slowly) creating a React wrapper library for Fine Uploader UI, so that is accounting for all of my FU dev time at the moment.
On top of that, I feel I need to make time to understand STS a bit better so that I am able to properly test support for this once the feature is complete. I see this as being a fair amount of work. Another dedicated maintainer would make this all go a bit faster.
it would be great to get the plumbing working with temporary credentials since this is arguably weakening the security to put static credentials on boxes
Completely agree. Unfortunately, there are quite a few other feature requests that have yet to be completed, in addition to this one. And, for the most part, I'm the only dedicated maintainer for this project. If this changes and someone else would be willing to come onto the project as a dedicated maintainer, we can make better progress. But being a maintainer is not a trivial amount of work, and requires an excellent understanding of HTML, CSS, JavaScript, HTTP, and S3, along with enough free time to make reasonable progress.
Thanks for the details @eugenebond. And @rnicholus no worries! Fineuploader's feature set and quality make it seem like a small army has been working on it, hard to believe it's just mostly you. Impressive, and I'm grateful. I can get pretty far using on-instance credentials, and when support for temporary credentials makes it's way in (I'm using serverside signing) we'll use it then.
Thanks for the kind words @trauts2. It's always good to hear that my work on this project has been helpful to users. My employer will likely need support for temporary credentials w/ a server-side signing workflow in the near future as well, so this feature will likely make it into the library in due course.
@mgoodness I'm guessing/hoping your knowledge of STS/AWS temp credentials is greater than mine. Perhaps we could coordinate on implementation of this feature, since @Widen will likely need this support for all of our their products at some point anyway.
@rnicholus I'd be thrilled to help with implementation. My current knowledge of STS is exclusively on the ops/provisioning side of things, but I'll devote some cycles to learning the API when I'm back in the office next week.
Cool, thanks Mike. I'm mostly interested in being sure that I understand how to use STS properly in a real-world scenario, using AWS best practices. Let's talk next week. If anyone else in this thread is interested in contributing information about their intended use of STS/temp credentials, please let me know ASAP. Perhaps we can all chat at once.
Count me in if you think I'd be helpful. I just did a bit of a deep dive into IAM instance profiles and at the very least can provide a working testbed
On Wednesday, June 29, 2016, Ray Nicholus [email protected] wrote:
Cool, thanks Mike. I'm mostly interested in being sure that I understand how to use STS properly in a real-world scenario, using AWS best practices. Let's talk next week. If anyone else in this thread is interested in contributing information about their intended use of STS/temp credentials, please let me know ASAP. Perhaps we can all chat at once.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/FineUploader/fine-uploader/issues/1406#issuecomment-229258369, or mute the thread https://github.com/notifications/unsubscribe/AD0TaMH5JsGSFtjze9Qac_Jyz53A8Dzwks5qQf9OgaJpZM4EQYP9 .
Sorry everyone, it is taking longer than I thought to simply take some time to write up a list of requirements for this feature. I promise I'll add something here in the next week. Stay tuned.
Hey @rnicholus - this issue doesn't look like it's moved in the past few months. Is there any more information you need or help the community can provide to push this forward? Thanks!
Mostly time. I won't have time to participate in any features of this size for the next few months or so due to commitments that are popping up.