Meteor-CollectionFS icon indicating copy to clipboard operation
Meteor-CollectionFS copied to clipboard

How to get URL of file inserted into S3 bucket?

Open derwaldgeist opened this issue 9 years ago • 5 comments

I managed to upload a file successfully to S3. But how do I get the URL to display this file in the browser?

According to the docs (which are quite hard to find, due to invalid links), it should be as easy as calling .url() on the returned fileObj. But this method always returns "null".

This is what I am doing:

  images.insert(filePath, function(error, fileObj) {
    if (error) {
      console.log('Sending file to S3 returned an error.');
      return;
    }
    console.log('Sending file to S3 was successful.');
    console.log(fileObj.url());

If I add the option brokenIsFine(), I get an URL like this:

/cfs/files/media/miqSYYRz4cMxcfu97

This seems to be a local URL, so I tried use it with my Meteor base URL. But this only returned the error

Access denied [403]

even after waiting for some time, so that the upload should have been finished in the meantime.

It would be really helpful if the documentation would state clearly how you can get an URL to a file that you have uploaded using CollectionFS. It took me a long time to even find the url() documentation in the .md files...

And I am still wondering if there is a way to get a direct Amazon S3 URL for my image?

derwaldgeist avatar Oct 08 '15 13:10 derwaldgeist

not having the url returned is a bit annoying. I guess the package needs some love. however, you can easily construct the url from what you know:

https://s3-[yourRegion].amazonaws.com/[yourBucket]/[yourCollectionName]/[yourFolderName]/[yourKeyName]

SierraGolf avatar Oct 27 '15 14:10 SierraGolf

Thanks for your feedback.

As you described, I ended up constructing the S3 URL's myself (after some more research on StackOverflow where some undocumented internal methods were posted). Yet, I would really appreciate if the the url() method would return the URL directly, as the internal implementation may change over time (and it actually did; I had to patch the solutions I found on StackOverflow to make them work with the most recent versions).

For reference, here's my current implementation, based on a StackOverflow post:

FS.File.prototype.S3Url = function(storeName) {
  var self = this;
  if (!self.isMounted()) {
    return null;
  }
  var collection = self.getCollection();
  var bucket = (collection.primaryStore || {}).bucket;
  if (storeName) {
    bucket = (collection.storesLookup[storeName] || {}).bucket || bucket;
  }
  if (!bucket) {
    return null;
  }
  var baseUrl = 'https://' + bucket + '.s3.amazonaws.com/';
  // Make sure the file object is mounted in a cfs
  var fileKey = self.collectionName + '/' + self._id + '-' + self.name();
  return baseUrl + fileKey;
};

This can be called on the fileObj that's passed to the insert() callback.

I also had to implement a poll mechanism to check when the file was actually available via this URL, as the URL won't work immediatly after the upload -- even if the upload was finished according to CollectionFS. Oviously, S3 takes some time for internal processing before you can actually download the file. Without my additional polling, the user would see a broken image link. So I'm polling the URL using HTTP HEAD requests.

derwaldgeist avatar Oct 28 '15 05:10 derwaldgeist

Super!

bmustata avatar Nov 02 '15 20:11 bmustata

nice one! I added a line to take into account the store base folder:

FS.File.prototype.S3Url = function(storeName) {
  var self = this;
  if (!self.isMounted()) {
    return null;
  }
  var collection = self.getCollection();
  var store = storeName ? collection.storesLookup[storeName] : collection.primaryStore || {};
  var bucket = store.bucket;
  var storeFolder = store.folder;
  if (!bucket) {
    return null;
  }
  var baseUrl = 'https://' + bucket + '.s3.amazonaws.com/';
  // Make sure the file object is mounted in a cfs
  var fileKey = storeFolder + '/' + self.collectionName + '/' + self._id + '-' + self.name();
  return baseUrl + fileKey;
};

team-pie avatar Jan 12 '16 23:01 team-pie

There are two approaches:

  1. Creating a pre-signed URL for private objects: https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURLJavaSDK.html
  2. getUrl method for public objects: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#getUrl-java.lang.String-java.lang.String-

harshachandra avatar May 18 '18 05:05 harshachandra