Upload multiple files using TransferUtility

[ersps25]

My requirement is to upload multiple images using AWS-SDK-ANDROID, I am using TransferUtility, but I didn't find any method to upload multiple images inside documentation of TransferUtility. The only suggestion I went through on stackoverflow was to create a loop and upload multiple images, it doesn't seem me right way to do this task. Please suggest which approach should I follow to upload multiple images.

[mutablealligator]

@ersps25 Thank you for reporting to us. This is a limitation of the TransferUtility. We will take this as a feature request to the team. At this time, please loop through the images and upload each individually.

[samyakjain]

How soon can we expect this feature?

[ShwetaChauhan18]

Is it possible now?

[vishalsgithub]

now it is possible or not..?

[ranjith-zen3]

even I'm facing a similar issue, do we have any update on this?

[Suyash171]

Is it possible now ?

[jpignata]

Thanks for your comments! No, we've not as of yet built this feature, but please continue to vote on it via reactions so we can properly prioritize it. For now, you should be able to upload multiple files by iterating through them (which is effectively what we'd do anyway if we were to build this feature) as suggesting in the initial issue.

[jamesonwilliams]

We will expose functionality for this, in the APIs, in the future.

For now, you could do something like this.

Firstly, some boiler-plate to obtain a TransferUtility instance:

Single<TransferUtility> transferUtility() {
    AWSConfiguration configuration = new AWSConfiguration(getApplicationContext());
    return s3Client(configuration).map(s3 ->
        TransferUtility.builder()
            .context(getApplicationContext())
            .s3Client(s3)
            .awsConfiguration(configuration)
            .build()
    );
}

Single<AmazonS3Client> s3Client(AWSConfiguration configuration) {
    return credentialsProvider().map(credentialsProvider -> {
        String regionInConfig = configuration
            .optJsonObject("S3TransferUtility")
            .getString("Region");
        Region region = Region.getRegion(regionInConfig);
        return new AmazonS3Client(credentialsProvider, region);
    });
}

Single<AWSMobileClient> credentialsProvider() {
    return Single.create(emitter -> {
        AWSMobileClient.getInstance().initialize(getApplicationContext(), new Callback<UserStateDetails>() {
            @Override
            public void onResult(UserStateDetails result) {
                emitter.onSuccess(AWSMobileClient.getInstance());
            }

            @Override
            public void onError(Exception error) {
                emitter.onError(error);
            }
        });
    });
}

Having that, a utility to upload multiple files, to multiple keys, and await the result:

Completable uploadMultiple(Map<File, String> fileToKeyUploads) {
    return transferUtility()
        .flatMapCompletable(transferUtility ->
            Observable.fromIterable(fileToKeyUploads.entrySet())
                .flatMapCompletable(entry -> uploadSingle(transferUtility, entry.getKey(), entry.getValue()))
        );
}

Completable uploadSingle(TransferUtility transferUtility, File aLocalFile, String toRemoteKey) {
    return Completable.create(emitter -> {
        transferUtility.upload(toRemoteKey, aLocalFile).setTransferListener(new TransferListener() {
            @Override
            public void onStateChanged(int id, TransferState state) {
                if (TransferState.FAILED.equals(state)) {
                    emitter.onError(new Exception("Transfer state was FAILED."));
                } else if (TransferState.COMPLETED.equals(state)) {
                    emitter.onComplete();
                }
            }

            @Override
            public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
            }

            @Override
            public void onError(int id, Exception exception) {
                emitter.onError(exception);
            }
        });
    });
}

Finally, use it:

private void example() {
    uploadMultiple(Collections.singletonMap("remote_key", new File()))
        .subscribeOn(Schedulers.io())
        .observeOn(Schedulers.io())
        .subscribe(() -> "Uploads completed!");
}

You could adapt it so that uploadMultiple ran the uploadSingle Completables in parallel.

[gaurav-likeminds]

@jamesonwilliams As per the solution proposed by you, how do I map multiple file uploading progress change? Is it viable to take out an average of all the files?

[lon9man]

@gaurav-likeminds i also need such functionality. i created issue https://github.com/aws-amplify/aws-sdk-android/issues/2390 but since creation didn't get any answer

[gaurav-likeminds]

@lon9man Right now I am using average of the progress of all the files getting uploaded. You can try to do that, it will give you an approx progress of the multi files as a whole.

[Fauzdar1]

Hey @gaurav-likeminds and @lon9man, I implemented the code provided by @jamesonwilliams in June, last year and it works with map. Your comment's notification reminded me, so, here's the Git repo I had created for this to share with others. Feel free to use it.

MultiUploaderS3 - Upload/Download multiple files - AWS