aws-sdk-ios icon indicating copy to clipboard operation
aws-sdk-ios copied to clipboard
verified publisher icon aws-amplify

No callback when uploading objects to S3 with SDK for iOS

Open temmplar47 opened this issue 5 years ago • 11 comments

Describe the bug When customer's application uploads objects to S3 with AWSS3TransferUtility, they sometimes got no callback (success or failure). The issue could lead to failure of their APP.

To Reproduce They could repro the issue by passing a null string to AWSToken by force. Here's the snippet of the code:

- (void)uploadUseFileName:(nonnull NSString *)fileName filePath:(nullable NSString *)filePath fileData:(nullable NSData *)fileData completionHandler:(AWSS3TransferUtilityUploadCompletionHandlerBlock)handler {
AWSS3TransferUtility *transferUtility = [AWSS3TransferUtility S3TransferUtilityForKey:@"AWSRegionCNNorth1"];
AWSS3TransferUtilityUploadExpression *expression = [[AWSS3TransferUtilityUploadExpression alloc] init];
[expression setValue:@"public-read" forRequestHeader:@"x-amz-acl"];

NSString *newFileName;
newFileName = fileName;
NSString *contentType = @"application/xml";
if ([fileName containsString:@".xml"]) {
contentType = @"application/xml";
} else if ([fileName containsString:@".mp3"] || [fileName containsString:@".spx"]) {
contentType = @"audio/mp3";
} else if ([fileName containsString:@".log"]) {
contentType = @"text/plain";
} else {
contentType = @"image/png";
}
if (!([fileName containsString:@"user/feedback/"] || [fileName containsString:@"user/photo/"])) {//不是意见反馈上传图片
NSString *leftFileName = [fileName hasSuffix:@".xml"] ? @"engine-result/" : @"ios/";
newFileName = [NSString stringWithFormat:@"%@%@",leftFileName, fileName];
}
WS(weakSelf);
if (filePath) {//The path
[transferUtility uploadFile:[NSURL fileURLWithPath:filePath] bucket:S3BucketName key:newFileName contentType:contentType expression:expression completionHandler:^(AWSS3TransferUtilityUploadTask * _Nonnull task, NSError * _Nullable error) {
handler(task, error);
if (error) {
[weakSelf uploadErrorToServer:error];
}
}];
} else if (fileData) { //Uploading data
[transferUtility uploadData:fileData bucket:S3BucketName key:newFileName contentType:contentType expression:expression completionHandler:^(AWSS3TransferUtilityUploadTask * _Nonnull task, NSError * _Nullable error) {
handler(task, error);
if (error) {
[weakSelf uploadErrorToServer:error];
}
}];
}
}

Which AWS service(s) are affected? S3

Expected behavior

Screenshots If applicable, add screenshots to help explain your problem.

Environment(please complete the following information):

  • SDK Version: 2.8.0, 2.9.6, 2.12.7
  • Dependency Manager: Cocoapods
  • Swift Version :

Device Information (please complete the following information):

  • Device:
  • iOS Version:
  • Specific to simulators:

Additional context

temmplar47 avatar Feb 27 '20 02:02 temmplar47

Could you provide us the following details?

  • How are you setting the AWSToken to null?
  • How are the credentials handled? Are you using AWSMobileClient?
  • Can you provide code sample on how the AWSS3TransferUtility is initialized

royjit avatar Feb 27 '20 02:02 royjit

  1. Achieved with an internal method of a child class of AWSCognitoCredentialsProviderHelper. The following method is used to provide token to SDK. If they set awsToken to a null string, the no callback issue will be reproed.

  • (AWSTask <NSString*> *) token { self.identityId = [UserManager sharedInstance].identityId; return [AWSTask taskWithResult:[UserManager sharedInstance].awsToken]; }

  • (AWSTask<NSDictionary<NSString *, NSString *> *> *)logins { self.identityId = [UserManager sharedInstance].identityId; return [AWSTask taskWithResult:@{@"XXXXXX" : [UserManager sharedInstance].awsToken }] ; }

  1. No. They referred to the integration method in the doc: https://docs.aws.amazon.com/zh_cn/cognito/latest/developerguide/developer-authenticated-identities.html
  2. //aws register with identity provider

  • (void)registerProvider { MineAWSCognitoCredentialsProviderHelper * devAuth = [[MineAWSCognitoCredentialsProviderHelper alloc] initWithRegionType:AWSRegionCNNorth1 identityPoolId:AWSanpoolId useEnhancedFlow:YES identityProviderManager:nil];

    AWSCognitoCredentialsProvider *credentialsProvider = [[AWSCognitoCredentialsProvider alloc] initWithRegionType:AWSRegionCNNorth1 identityProvider:devAuth];

    AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionCNNorth1 credentialsProvider:credentialsProvider];

    [AWSS3TransferUtility registerS3TransferUtilityWithConfiguration:configuration forKey:@"AWSRegionCNNorth1"]; }

temmplar47 avatar Feb 27 '20 09:02 temmplar47

Thanks for the information. We are trying to reproduce this issue.

Other related issues #2245, #2211

royjit avatar Feb 28 '20 05:02 royjit

If a credential provider returns null credentials, I would not expect the upload to even start, let alone fire a callback method. Because all TransferUtility activity happens in background NSURLSessions, it uses presigned S3 URLs to upload. These URLs are created at the time the upload is initiated, and the process of creating those presigned URLs require valid AWS credentials.

Some questions:

  • Are you seeing the content upload successfully to S3 (e.g., by inspecting the target bucket in the S3 console), even though the callback is not invoked? Or is it the case that the upload does not complete?
  • Do you have logging enabled? If not, please enable it and let us know what you see. I'm especially interested in any errors you see when passing null tokens to the creation method, but it could also be an error when the callback fails to invoke after an otherwise successful upload. 
    [[AWSDDLog sharedInstance] setLogLevel:AWSDDLogLevelVerbose];
[AWSDDLog addLogger:[AWSDDTTYLogger sharedInstance]];
  • Does this happen only if the app is in the background, or also if it is in foreground? If only in background, are you intercepting background url events in handleEventsForBackgroundURLSession, as described in the background transfer docs?

In the meantime, I'm attempting to reproduce this issue with the S3 Transfer Utility sample app.

palpatim avatar Feb 28 '20 20:02 palpatim

  1. Unsuccessful;
  2. PFA [超长消息]Response he.pdf
  3. Both in background & foreground.

temmplar47 avatar Mar 03 '20 09:03 temmplar47

Can you post a log snippet; I am unable to interpret the document linked in the 2nd bullet.

palpatim avatar Mar 04 '20 06:03 palpatim

Response headers:

{
 Connection = "keep-alive";
 "Content-Length" = 312;
 "Content-Type" = "application/x-amz-json-1.1";
 Date = "Tue, 03 Mar 2020 08:47:10 GMT";
 "x-amzn-ErrorMessage" = "1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint: Map value
must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]";
 "x-amzn-ErrorType" = "ValidationException:";
 "x-amzn-RequestId" = "92032d6c-5d2b-11ea-aa9c-37e769583e0c";
}
2020-03-03 16:47:10:481 IELTSJunior[15873:2818695] Response body:
{"__type":"ValidationException","message":"1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint:
Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]"}
2020-03-03 16:47:10:481 IELTSJunior[15873:2818695] Unable to refresh. Error is [Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain
Code=2 "GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity
configuration.}]
2020-03-03 16:47:10:481 IELTSJunior[15873:2818695] Error: Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain Code=2
"GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration.}
2020-03-03 16:47:10:482 IELTSJunior[15873:2818690] Request headers:
{
 "Content-Type" = "application/x-amz-json-1.1";
 Host = "cognito-identity.cn-north-1.amazonaws.com.cn";
 "User-Agent" = "aws-sdk-iOS/2.12.7 iOS/13.3 zh_CN";
 "X-Amz-Date" = 20200303T084710Z;
 "X-Amz-Target" = "AWSCognitoIdentityService.GetCredentialsForIdentity";
}
2020-03-03 16:47:10:482 IELTSJunior[15873:2818690] Request body:
{"Logins":{"cognito-identity.cn-north-1.amazonaws.com.cn":""},"IdentityId":"cn-north-1:4b046610-78bc-48f9-a9e0-aa4d8168ff37"}
2020-03-03 16:47:10:536 IELTSJunior[15873:2818118] Response headers:
{
 Connection = "keep-alive";
 "Content-Length" = 312;
 "Content-Type" = "application/x-amz-json-1.1";
 Date = "Tue, 03 Mar 2020 08:47:10 GMT";
 "x-amzn-ErrorMessage" = "1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint: Map value
must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]";
 "x-amzn-ErrorType" = "ValidationException:";
 "x-amzn-RequestId" = "9223adfc-5d2b-11ea-ba8c-cf403c501aff";
}
2020-03-03 16:47:10:536 IELTSJunior[15873:2818118] Response body:
{"__type":"ValidationException","message":"1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint:
Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]"}
2020-03-03 16:47:10:536 IELTSJunior[15873:2818118] GetCredentialsForIdentity failed. Error is [Error Domain=com.amazonaws.AWSCognitoIdentityErrorDomain
Code=0 "(null)" UserInfo={__type=ValidationException, message=1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed
to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]}]
2020-03-03 16:47:10:537 IELTSJunior[15873:2818118] Resetting identity Id and calling getIdentityId
2020-03-03 16:47:10:549 IELTSJunior[15873:2818118] Retrying GetCredentialsForIdentity
2020-03-03 16:47:10:549 IELTSJunior[15873:2818118] Request headers:
{
 "Content-Type" = "application/x-amz-json-1.1";
 Host = "cognito-identity.cn-north-1.amazonaws.com.cn";
 "User-Agent" = "aws-sdk-iOS/2.12.7 iOS/13.3 zh_CN";
 "X-Amz-Date" = 20200303T084710Z;
 "X-Amz-Target" = "AWSCognitoIdentityService.GetCredentialsForIdentity";
}
2020-03-03 16:47:10:549 IELTSJunior[15873:2818118] Request body:
{"Logins":{"cognito-identity.cn-north-1.amazonaws.com.cn":""},"IdentityId":"cn-north-1:4b046610-78bc-48f9-a9e0-aa4d8168ff37"}
2020-03-03 16:47:10:620 IELTSJunior[15873:2818708] Response headers:
{
 Connection = "keep-alive";
 "Content-Length" = 312;
 "Content-Type" = "application/x-amz-json-1.1";
 Date = "Tue, 03 Mar 2020 08:47:10 GMT";
 "x-amzn-ErrorMessage" = "1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint: Map value
must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]";
 "x-amzn-ErrorType" = "ValidationException:";
 "x-amzn-RequestId" = "922b00b7-5d2b-11ea-961a-5d3514bbee32";
}
2020-03-03 16:47:10:621 IELTSJunior[15873:2818708] Response body:
{"__type":"ValidationException","message":"1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint:
Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]"}
2020-03-03 16:47:10:621 IELTSJunior[15873:2818708] Unable to refresh. Error is [Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain
Code=2 "GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity
configuration.}]
2020-03-03 16:47:10:621 IELTSJunior[15873:2818708] Error: Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain Code=2
"GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration.}
2020-03-03 16:47:10:622 IELTSJunior[15873:2818414] Request headers:
{
 "Content-Type" = "application/x-amz-json-1.1";
 Host = "cognito-identity.cn-north-1.amazonaws.com.cn";
 "User-Agent" = "aws-sdk-iOS/2.12.7 iOS/13.3 zh_CN";
 "X-Amz-Date" = 20200303T084710Z;
 "X-Amz-Target" = "AWSCognitoIdentityService.GetCredentialsForIdentity";
}
2020-03-03 16:47:10:622 IELTSJunior[15873:2818414] Request body:
{"Logins":{"cognito-identity.cn-north-1.amazonaws.com.cn":""},"IdentityId":"cn-north-1:4b046610-78bc-48f9-a9e0-aa4d8168ff37"}
2020-03-03 16:47:10:704 IELTSJunior[15873:2818708] Response headers:
{
 Connection = "keep-alive";
 "Content-Length" = 312;
 "Content-Type" = "application/x-amz-json-1.1";
 Date = "Tue, 03 Mar 2020 08:47:09 GMT";
 "x-amzn-ErrorMessage" = "1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint: Map value
must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]";
 "x-amzn-ErrorType" = "ValidationException:";
 "x-amzn-RequestId" = "923846e5-5d2b-11ea-a5a3-cbc8090014bd";
}
2020-03-03 16:47:10:704 IELTSJunior[15873:2818708] Response body:
{"__type":"ValidationException","message":"1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint:
Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]"}
2020-03-03 16:47:10:704 IELTSJunior[15873:2818708] GetCredentialsForIdentity failed. Error is [Error Domain=com.amazonaws.AWSCognitoIdentityErrorDomain
Code=0 "(null)" UserInfo={__type=ValidationException, message=1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed
to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]}]
2020-03-03 16:47:10:704 IELTSJunior[15873:2818708] Resetting identity Id and calling getIdentityId
2020-03-03 16:47:10:737 IELTSJunior[15873:2818708] Retrying GetCredentialsForIdentity
2020-03-03 16:47:10:738 IELTSJunior[15873:2818708] Request headers:
{
 "Content-Type" = "application/x-amz-json-1.1";
 Host = "cognito-identity.cn-north-1.amazonaws.com.cn";
 "User-Agent" = "aws-sdk-iOS/2.12.7 iOS/13.3 zh_CN";
 "X-Amz-Date" = 20200303T084710Z;
 "X-Amz-Target" = "AWSCognitoIdentityService.GetCredentialsForIdentity";
}
2020-03-03 16:47:10:738 IELTSJunior[15873:2818708] Request body:
{"Logins":{"cognito-identity.cn-north-1.amazonaws.com.cn":""},"IdentityId":"cn-north-1:4b046610-78bc-48f9-a9e0-aa4d8168ff37"}
2020-03-03 16:47:10:787 IELTSJunior[15873:2818695] Response headers:
{
 Connection = "keep-alive";
 "Content-Length" = 312;
 "Content-Type" = "application/x-amz-json-1.1";
 Date = "Tue, 03 Mar 2020 08:47:09 GMT";
 "x-amzn-ErrorMessage" = "1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint: Map value
must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]";
 "x-amzn-ErrorType" = "ValidationException:";
 "x-amzn-RequestId" = "9247fef5-5d2b-11ea-9808-0bd493bc88f8";
}
2020-03-03 16:47:10:787 IELTSJunior[15873:2818695] Response body:
{"__type":"ValidationException","message":"1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint:
Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]"}
2020-03-03 16:47:10:788 IELTSJunior[15873:2818695] Unable to refresh. Error is [Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain
Code=2 "GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity
configuration.}]
2020-03-03 16:47:10:788 IELTSJunior[15873:2818695] Error: Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain Code=2
"GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration.}

temmplar47 avatar Mar 05 '20 00:03 temmplar47

As noted in the error log you posted, the issue appears to be in your app's authentication configuration:

{"__type":"ValidationException","message":"1 validation error detected: Value '{cognito-identity.cn-north-1.amazonaws.com.cn=}' at 'logins' failed to satisfy constraint:
Map value must satisfy constraint: [Member must have length less than or equal to 50000, Member must have length greater than or equal to 1]"}
2020-03-03 16:47:10:481 IELTSJunior[15873:2818695] Unable to refresh. Error is [Error Domain=com.amazonaws.AWSCognitoCredentialsProviderErrorDomain
Code=2 "GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity configuration."
UserInfo={NSLocalizedDescription=GetCredentialsForIdentity keeps failing. Clearing identityId did not help. Please check your Amazon Cognito Identity
configuration.}]

You'll need to investigate your app's login setup (you mentioned you were using developer authenticated identities) to see if you can isolate the problem. For example, you could try reducing your app to its simplest components, with no dependencies other than those necessary to perform a sign-in using your developer authenticated identity setup, then invoke the -[credentialsProvider credentials] method and see if it works. That way you remove S3 from the equation altogether and focus on just the authentication piece. If you're able to retrieve credentials that way, then investigate the coupling between your custom credentials provider and the rest of the system, to make sure that it is vending the correct credentials when TransferUtility asks for them.

palpatim avatar Mar 05 '20 14:03 palpatim

The customer reopened the case and would need a solution to empty or wrong tokens. Their app would produce empty or wrong tokens and does need a callback. Can you provide a sample solution to this?

temmplar47 avatar Mar 30 '20 06:03 temmplar47

We will investigate the behavior of the TransferUtility callback if the credential provider returns incorrect tokens.

palpatim avatar Apr 02 '20 18:04 palpatim

Could you please confirm if iOS SDK calls clearCredentials to refresh credentials when they have expired, like Android calling the refresh method mentioned in the doc? Could you please provide any samples?

temmplar47 avatar Apr 07 '20 09:04 temmplar47