Storage.put(...{level: 'private'}) returns uploaded file key without prefix

[sorokinvj]

Describe the bug It is not a bug I believe, but Storage.put(...{level: 'private'}) returns uploaded file key without prefix.

To Reproduce Steps to reproduce the behavior:

const = 'key1'
const uploadedFile = await Storage.put(key, file, {
          level: 'private',
          contentType: mimeType,
          progressCallback(progress) {
            setProgress((progress.loaded / progress.total) * 100)
          },
        })
console.log(uploadedFile)

Storage.put returns { key: 'key1' } but it should be { key: 'private/{region}:{cognitoID}/key1} for private files:

Expected behavior Key returned with the prefix (private folder)

  System:
    OS: macOS Mojave 10.14.6    CPU: (4) x64 Intel(R) Core(TM) i5-8210Y CPU @ 1.60GHz
    Memory: 60.18 MB / 8.00 GB    Shell: 3.2.57 - /bin/bash
  Binaries:    Node: 12.6.0 - ~/.nvm/versions/node/v12.6.0/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn    npm: 6.9.0 - ~/.nvm/versions/node/v12.6.0/bin/npm
  Browsers:    Chrome: 81.0.4044.138
    Firefox: 74.0    Safari: 13.1
  npmPackages:    @alfonmga/react-lottie-light-ts: ^0.0.1 => 0.0.1     @babel/core: ^7.6.0 => 7.8.3     @lingui/cli: ^2.9.1 => 2.9.1 
    @lingui/loader: ^2.9.1 => 2.9.1 
    @lingui/macro: ^2.8.3 => 2.9.0 
    @lingui/react: ^2.9.0 => 2.9.1 
    @stripe/react-stripe-js: ^1.1.2 => 1.1.2 
    @stripe/stripe-js: ^1.4.0 => 1.4.0 
    amr-duration: ^0.2.0 => 0.2.0 
    aws-amplify: ^1.1.40 => 1.3.3 
    aws-amplify-react: ^2.5.3 => 2.6.3 
    axios: ^0.19.2 => 0.19.2 
    babel-core: 7.0.0-bridge.0 => 7.0.0-bridge.0 
    babel-plugin-dynamic-import-node: ^2.3.0 => 2.3.0 
    babel-plugin-styled-components: ^1.10.0 => 1.10.6 
    babel-plugin-syntax-dynamic-import: ^6.18.0 => 6.18.0 
    babel-plugin-transform-class-properties: ^6.24.1 => 6.24.1 
    dayjs: ^1.8.13 => 1.8.19 
    docx: ^5.0.2 => 5.0.2 
    eslint-config-prettier: ^6.7.0 => 6.9.0 
    eslint-plugin-import: ^2.14.0 => 2.20.0 
    eslint-plugin-jsx-a11y: ^6.1.1 => 6.2.3 
    eslint-plugin-prettier: ^3.1.2 => 3.1.2 
    eslint-plugin-react: ^7.11.0 => 7.17.0 
    esm: ^3.2.22 => 3.2.25 
    file-saver: ^2.0.2 => 2.0.2 
    jest: ^24.9.0 => 24.9.0 
    music-metadata-browser: ^2.0.2 => 2.0.2 
    prettier: ^1.19.1 => 1.19.1 
    prop-types: ^15.7.2 => 15.7.2 
    react: ^16.8.2 => 16.12.0 
    react-dom: ^16.8.2 => 16.12.0 
    react-dropzone: ^10.1.5 => 10.2.1 
    react-parallax: ^3.0.3 => 3.0.3 
    react-router: ^5.0.1 => 5.1.2 
    react-router-dom: ^5.0.1 => 5.1.2 
    react-scripts: latest => 3.3.0 
    semantic-ui-react: latest => 0.88.2 
    slate: ^0.57.1 => 0.57.1 
    slate-react: ^0.57.1 => 0.57.1 
    styled-components: ^4.1.3 => 4.4.1 
    uuid: ^3.4.0 => 3.4.0 
    wavesurfer.js: ^3.0.0 => 3.3.1 
  npmGlobalPackages:    @aws-amplify/cli: 4.11.0
    esm: 3.2.25
    nodemon: 2.0.0
    npm: 6.9.0
    pm2: 4.1.2
    serve: 11.2.0
    serverless: 1.51.0
    ts-node: 8.9.0
    typescript: 3.8.3```

Additional context I am building speech recognition app:

1. User uploads private file to s3
2. User hits button Transcribe in UI, that action calls API and triggers Lambda that starts transcription job in AWSTranscribe.

With public files its easy - just append /public/ to s3 key and this would be correct path to the file. With private files its more complex – I need to know cognito id of the user that uploads the file. It would be great if Storage.put just returned this path for private files by default

[sammartinez]

@sorokinvj Apologizes for taking this long to respond on this. Can I ask, are you able to use a later version of aws-amplify? I am seeing you on version 1 which we are on version 3. Please let us know.

[stale[bot]]

This issue has been automatically closed because of inactivity. Please open a new issue if are still encountering problems.

[SethThomas]

The issue is not unique to private file storage only. The issue also pops up when protected file levels are used. This has to do with how Amplify implements file access levels. Amplify creates a prefix when uploading a file based on the file access level: https://github.com/aws-amplify/amplify-js/blob/main/packages/storage/src/providers/AWSS3Provider.ts#L253.

I have a fairly simple helper method to upload a file to S3 using Amplify from my client application:

export async function s3Upload(file) {
  const filename = `${Date.now()}-${file.name}`;
  const stored = await Storage.put(filename, file, {
    contentType: file.type,
    level: "protected"
  });
  return stored.key;
}

When I call this method with a file named my_file.jpg, Amplify stores the file in <my_bucket>/protected/<cognitoIdentityId>/<timestamp>-my_file.jpg. However, the Storage.put method returns a key of <timestamp>-my_file.jpg, which is not the full object key.

This is problematic, as I'm storing the object key in DynamoDB for later retrieval via Cloudfront. This requires me to know the full object key:

export function getCloudfrontUrl(keyName, bucketName) {
  let imageRequest = JSON.stringify({
    bucket: bucketName,
    key: keyName.  // < ---- this needs to be `<my_bucket>/protected/<cognitoIdentityId>/<timestamp>-my_file.jpg`
  });
  return `https://<my-cloudfront-url>.cloudfront.net/${btoa(imageRequest)}`;
}

It would be nice if the Storage module returned the full object key instead of just the filename. Absent this feature, my client code has to know about Amplify implementation details to rebuild the full object key. I'm OK with Amplify storing the object however it wants, I'd just prefer to get the full object key handed back to my client code.

[Mac-prof]

I face a similar issue (Angular, typescript) - did you already solved yours and what was the root cause?

Situation: I have pretty standard Amplify project leveraging on Angular, Cognito authentication, Dynamo DB for tables storage and S3 for images storage. I want to have one of my Dynamo DB fields to point a key to the PRIVATE bucket, that will let me generate URL for image to be presented on the web page.

Complication: the Storage.put function stores properly the image file in the private bucket, putting a dedicated prefix per each Cognito user (so sort of folder). I face however a problem to programmatically get the full Key of the stored image file. The put function returns only the key that contains the file name instead of prefix + file name in a given private bucket.

Would you please suggest what do I do wrongly? Thank you in advance.

While storing browser selected image I put this this way: const response = await Storage.put("NewImageName", "NewImageFile", { level: 'private', contentType: 'image/jpg', } .then (result => { console.log('result string' + result); }) .catch(err => console.log(err)); } It is stored in a private bucket with some prefix generated by a put function specifically for given Cognito user.

When trying to retrieve the file I need to put the key... but the key I get from the put function does not contain a prefix - it only contains the actual file name (e.g. picture.jpg), so the url does not point the stored file properly. params = { Bucket: 'mybucketname', Key: key_obtained_from_put_function, Expires: 120*60 }; } var url = this.s3.getSignedUrl('getObject', params);

[undefobj]

We should track this as an enhancement

[HarborneD]

it would be great to get the full prefix back in the put response, but for now, what is the intended work around for accessing the file outside of amplify.

I can store the file name and user id of the owner in to DDB but it seems the user id accessible at the time of file upload doesn't correlate to the one used as the file prefix ?

[cfbo]

Any update on this? There are similar requests on Discord: https://discord.com/channels/705853757799399426/707328986077855836/902934952633573438 In my case it would be useful to get a result containing the following info: bucket: string, key: string, region: string, filename: string,

[dsws]

As a workaround, locally I use this command to find the private files via cli

aws s3 ls s3://s3-bucket-name/private/ --recursive | grep cabc1f14-0990-42e0-a9b6-2ec26143432a* | cut -c 32-

I guess you can write some lambda endpoint to do this call and return you the full file paths. I wish there was an easier way too.

[gardner]

If you came here looking for a workaround then you can use something like this:

import React, { useEffect, useState } from "react";
import { useAuthenticator } from '@aws-amplify/ui-react';
import { Auth } from 'aws-amplify';

function App() {
  const { authStatus } = useAuthenticator(context => [context.authStatus]);
  const [identityId, setIdentityId] = useState('');

  useEffect(() => {
    if (authStatus === 'authenticated') {
      Auth.currentUserCredentials().then((creds) => {
        setIdentityId(creds.identityId);
      });
    }
  }, [authStatus]);

  return (
    <span>{identityId}</span>
  );
}

export default App;

[arantespp]

If you need to use the working around with Amplify v6 (Auth.currentUserCredentials (DEPRECATED) is now fetchAuthSession):

import React, { useEffect, useState } from "react";
import { useAuthenticator } from '@aws-amplify/ui-react';
import { fetchAuthSession } from 'aws-amplify/auth';

function App() {
  const { authStatus } = useAuthenticator(context => [context.authStatus]);
  const [identityId, setIdentityId] = useState('');

  useEffect(() => {
    if (authStatus === 'authenticated') {
      fetchAuthSession().then((creds) => {
        setIdentityId(creds.identityId);
      });
    }
  }, [authStatus]);

  return (
    <span>{identityId}</span>
  );
}

export default App;