Authorization header in large file upload slices causing failures

[qiurunxing]

Expected behavior

We've been using this feature for a long time and in recent days we've noticed that some accounts can't upload files, but some do. We expect file uploads to work for all accounts.

microsoft-graph: 2.10.0

Actual behavior

com.microsoft.graph.core.ClientException: Upload session failed.
	at com.microsoft.graph.requests.extensions.ChunkedUploadRequest.upload(ChunkedUploadRequest.java:116)
	at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:186)
	at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:214)
	at com.cloudcoupler.OneDriveProvider$openDocument$1.invoke(OneDriveProvider.kt:457)
	at com.cloudcoupler.OneDriveProvider$openDocument$1.invoke(OneDriveProvider.kt:400)
	at com.cloudcoupler.UtilsKt$async$1$1.invokeSuspend(Utils.kt:26)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: com.microsoft.graph.core.ClientException: Error code: unauthenticated
Error message: Unauthenticated

PUT https://my.microsoftpersonalcontent.com/personal/a42fa85f95cc1a8f/_api/v2.0/drive/items/01ZSWHIXDZ6RIIRCR4D5BLTIFDIFQMLPFH/uploadSession?guid=%27a0589390-c436-4350-9102-854e9917aa9f%27&dc=0&tempauth=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBfZGlzcGxheW5hbWUiOiJHcmFwaCIsImFwcGlkIjoiMDAwMDAwMDMtMDAwMC0wMDAwLWMwMDAtMDAwMDAwMDAwMDAwIiwiYXVkIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwL215Lm1pY3Jvc29mdHBlcnNvbmFsY29udGVudC5jb21AOTE4ODA0MGQtNmM2Ny00YzViLWIxMTItMzZhMzA0YjY2ZGFkIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MDAwNjAwMDAyODhhNzBiM0BsaXZlLmNvbSIsImNpZCI6Iis4WUR2dzZjN1VLelVMU3BNNmNCQlE9PSIsImVuZHBvaW50dXJsIjoiVDFMRVhSUnAxVnZTczVJUThYSEhTa0hjc1ErZExvZyt0UzAxdHJyc3ZZST0iLCJlbmRwb2ludHVybExlbmd0aCI6IjE4NSIsImV4cCI6IjE3MTcyMjU0NTkiLCJpcGFkZHIiOiI1Mi4xMDQuNTguMTUwIiwiaXNsb29wYmFjayI6IlRydWUiLCJpc3MiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAiLCJuYmYiOiIxNzE3MTM5MDU5IiwicHVpZCI6IjAwMDYwMDAwMjg4QTcwQjMiLCJzY3AiOiJteWZpbGVzLnJlYWQgYWxsZmlsZXMud3JpdGUgYWxscHJvZmlsZXMucmVhZCIsInNpZCI6IjE3MjUwNjI0MjkwNDUwOTQ4MDE1XzMyMmZiNGMyLWUyYzgtNDdjZi04YmZiLWY5YzEwYjc5OTQxYiIsInNpdGVpZCI6Ik9ERm1aVEUwTkdFdE5tUTNaUzAwT0RrM0xUaGlORGt0Tm1Gak4yRXpPV1JrWWpKaiIsInRpZCI6IjkxODgwNDBkLTZjNjctNGM1Yi1iMTEyLTM2YTMwNGI2NmRhZCIsInR0IjoiMiIsInVwbiI6IndpdHR5dGVzdDAxQGhvdG1haWwuY29tIiwidmVyIjoiaGFzaGVkcHJvb2Z0b2tlbiJ9.g2cQ9l8ILGXETIe3DtSwvPvh41NtIiGktf2gUN69Uco
SdkVersion : graph-java/v2.10.0
Content-Range : bytes 0-18264/18265
Authorization : [PII_REDACTED]


401 : FORBIDDEN
Cache-Control : private, max-age=0
Content-Length : 64
Content-Security-Policy : frame-ancestors 'self' teams.microsoft.com *.teams.microsoft.com *.skype.com *.teams.microsoft.us local.teams.office.com *.office365.com goals.cloud.microsoft *.powerapps.com *.yammer.com engage.cloud.microsoft word.cloud.microsoft excel.cloud.microsoft powerpoint.cloud.microsoft *.officeapps.live.com *.office.com *.microsoft365.com *.stream.azure-test.net *.microsoftstream.com *.dynamics.com *.microsoft.com onedrive.live.com *.onedrive.live.com securebroker.sharepointonline.com;
Content-Type : application/json
Date : Fri, 31 May 2024 07:04:20 GMT
Expires : Thu, 16 May 2024 07:04:20 GMT
Last-Modified : Fri, 31 May 2024 07:04:20 GMT
MicrosoftSharePointTeamServices : 16.0.0.24908
MS-CV : qhkFo+d/Y0G1W+ZnjFzBmw.0
P3P : CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"
request-id : a30519aa-7fe7-4163-b55b-e6678c5cc19b
SPRequestGuid : a30519aa-7fe7-4163-b55b-e6678c5cc19b
Strict-Transport-Security : max-age=31536000
Vary : Origin

Steps to reproduce the behavior

Only some accounts can reproduce, but we noticed that the URL uploaded by the abnormal accounts is https://my.microsoftpersonalcontent.com/personal, and the upload URL used by the available accounts is https://api.onedrive.com/rup/, and the uploaded URL is from UploadSession. I'm a little curious as to why there is such a difference.

[Ndiritu]

Hi @qiurunxing

Thanks for raising this issue and the detailed logs.

This seems to be a change on the API side causing this and not a change on the SDK. If it's ok with you, may I redirect you to post this issue on Microsoft Q&A so that the OneDrive API team looks into this.

We also have a new version of the SDK 6.x that you can consider upgrading to for better support on SDK issues.

[qiurunxing]

Hi @Ndiritu

Thank you for your quick reply.

Please help redirect to the API team, thanks! We're also considering upgrading the SDK version, as you know, because the interface is quite different, and we need some time.

[Ndiritu]

@qiurunxing created the issue on Q&A for you. You'd need to sign-in and follow question to get updates and respond to any follow-up questions from the support team.

[qiurunxing]

Thanks @Ndiritu. Looks like the issue has been deleted. Can I create the issue on it myself ?

[Ndiritu]

@qiurunxing I'm not sure why it's been taken down. But yes, you can create the issue yourself so that you get notified about responses and requests for more info.

[qiurunxing]

Thanks @Ndiritu. I created a issue on that, Do you know who I can ping so I can get a faster reply? https://learn.microsoft.com/en-us/answers/questions/1691263/some-accounts-cannot-upload-files

[damiann5f]

We faced the same issue, which turned out to be caused by including an authorization header in the upload task.

As per docs:

If you include the Authorization header when issuing the PUT call, it may result in an HTTP 401 Unauthorized response. Only send the Authorization header and bearer token when issuing the POST during the first step. Don't include it when you issue the PUT call.

So, the solution can be to create a separate GraphServiceClient with an AuthenticationProvider that does nothing and to pass its RequestAdapter when creating a LargeFileUploadTask.

[Ndiritu]

Thank you for pointing this out @DamianNowak5f.

An alternative work-around is to set the requestAdapter property to null or initialize a GraphServiceClient with an AnonymousAuthenticationProvider.

Adding this to our backlog to make this experience better.

[andrueastman]

I believe this should be resolved with the latest version of the SDK and can be closed.

With the AzureIdentityAccessTokenProvider the SDK will prevent sending access tokens to non graph URLs(as in this case) to avoid this scenario and meet the requirements for the upload to not have an Auth header.

• https://github.com/microsoftgraph/msgraph-sdk-java-core/blob/997a098179d377cea5a687068fb6df55fa610cd0/src/main/java/com/microsoft/graph/core/authentication/AzureIdentityAccessTokenProvider.java#L22
• https://learn.microsoft.com/en-us/graph/api/driveitem-createuploadsession?view=graph-rest-1.0#remarks

[qiurunxing]

Thanks @andrueastman @DamianNowak5f @Ndiritu

But I'm curious that this issue is only reproduce on some accounts.

Only some accounts can reproduce, but we noticed that the URL uploaded by the abnormal accounts is "https://my.microsoftpersonalcontent.com/personal....", and the upload URL used by the available accounts is "https://api.onedrive.com/rup/....", and the upload URL is from UploadSession.

[andrueastman]

Its possible that one account is a MSA(personal) account while the other are school/work account. OneDrive Apis do behave differently based on the account types due to various reasons.

I believe the right thing to do is to always ensure the Auth header is not set. As the URL is not a graph API URL so the token would be invalid either way as the host is different from the token issuer, and the request should ideally fail.

[Hamza1122H]

Expected behavior

We've been using this feature for a long time and in recent days we've noticed that some accounts can't upload files, but some do. We expect file uploads to work for all accounts.

microsoft-graph: 2.10.0

Actual behavior

com.microsoft.graph.core.ClientException: Upload session failed.
	at com.microsoft.graph.requests.extensions.ChunkedUploadRequest.upload(ChunkedUploadRequest.java:116)
	at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:186)
	at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:214)
	at com.cloudcoupler.OneDriveProvider$openDocument$1.invoke(OneDriveProvider.kt:457)
	at com.cloudcoupler.OneDriveProvider$openDocument$1.invoke(OneDriveProvider.kt:400)
	at com.cloudcoupler.UtilsKt$async$1$1.invokeSuspend(Utils.kt:26)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: com.microsoft.graph.core.ClientException: Error code: unauthenticated
Error message: Unauthenticated

PUT https://my.microsoftpersonalcontent.com/personal/a42fa85f95cc1a8f/_api/v2.0/drive/items/01ZSWHIXDZ6RIIRCR4D5BLTIFDIFQMLPFH/uploadSession?guid=%27a0589390-c436-4350-9102-854e9917aa9f%27&dc=0&tempauth=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBfZGlzcGxheW5hbWUiOiJHcmFwaCIsImFwcGlkIjoiMDAwMDAwMDMtMDAwMC0wMDAwLWMwMDAtMDAwMDAwMDAwMDAwIiwiYXVkIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwL215Lm1pY3Jvc29mdHBlcnNvbmFsY29udGVudC5jb21AOTE4ODA0MGQtNmM2Ny00YzViLWIxMTItMzZhMzA0YjY2ZGFkIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MDAwNjAwMDAyODhhNzBiM0BsaXZlLmNvbSIsImNpZCI6Iis4WUR2dzZjN1VLelVMU3BNNmNCQlE9PSIsImVuZHBvaW50dXJsIjoiVDFMRVhSUnAxVnZTczVJUThYSEhTa0hjc1ErZExvZyt0UzAxdHJyc3ZZST0iLCJlbmRwb2ludHVybExlbmd0aCI6IjE4NSIsImV4cCI6IjE3MTcyMjU0NTkiLCJpcGFkZHIiOiI1Mi4xMDQuNTguMTUwIiwiaXNsb29wYmFjayI6IlRydWUiLCJpc3MiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAiLCJuYmYiOiIxNzE3MTM5MDU5IiwicHVpZCI6IjAwMDYwMDAwMjg4QTcwQjMiLCJzY3AiOiJteWZpbGVzLnJlYWQgYWxsZmlsZXMud3JpdGUgYWxscHJvZmlsZXMucmVhZCIsInNpZCI6IjE3MjUwNjI0MjkwNDUwOTQ4MDE1XzMyMmZiNGMyLWUyYzgtNDdjZi04YmZiLWY5YzEwYjc5OTQxYiIsInNpdGVpZCI6Ik9ERm1aVEUwTkdFdE5tUTNaUzAwT0RrM0xUaGlORGt0Tm1Gak4yRXpPV1JrWWpKaiIsInRpZCI6IjkxODgwNDBkLTZjNjctNGM1Yi1iMTEyLTM2YTMwNGI2NmRhZCIsInR0IjoiMiIsInVwbiI6IndpdHR5dGVzdDAxQGhvdG1haWwuY29tIiwidmVyIjoiaGFzaGVkcHJvb2Z0b2tlbiJ9.g2cQ9l8ILGXETIe3DtSwvPvh41NtIiGktf2gUN69Uco
SdkVersion : graph-java/v2.10.0
Content-Range : bytes 0-18264/18265
Authorization : [PII_REDACTED]


401 : FORBIDDEN
Cache-Control : private, max-age=0
Content-Length : 64
Content-Security-Policy : frame-ancestors 'self' teams.microsoft.com *.teams.microsoft.com *.skype.com *.teams.microsoft.us local.teams.office.com *.office365.com goals.cloud.microsoft *.powerapps.com *.yammer.com engage.cloud.microsoft word.cloud.microsoft excel.cloud.microsoft powerpoint.cloud.microsoft *.officeapps.live.com *.office.com *.microsoft365.com *.stream.azure-test.net *.microsoftstream.com *.dynamics.com *.microsoft.com onedrive.live.com *.onedrive.live.com securebroker.sharepointonline.com;
Content-Type : application/json
Date : Fri, 31 May 2024 07:04:20 GMT
Expires : Thu, 16 May 2024 07:04:20 GMT
Last-Modified : Fri, 31 May 2024 07:04:20 GMT
MicrosoftSharePointTeamServices : 16.0.0.24908
MS-CV : qhkFo+d/Y0G1W+ZnjFzBmw.0
P3P : CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"
request-id : a30519aa-7fe7-4163-b55b-e6678c5cc19b
SPRequestGuid : a30519aa-7fe7-4163-b55b-e6678c5cc19b
Strict-Transport-Security : max-age=31536000
Vary : Origin

Steps to reproduce the behavior

Only some accounts can reproduce, but we noticed that the URL uploaded by the abnormal accounts is https://my.microsoftpersonalcontent.com/personal, and the upload URL used by the available accounts is https://api.onedrive.com/rup/, and the uploaded URL is from UploadSession. I'm a little curious as to why there is such a difference.

[Hamza1122H]

To https://github.com/microsoftgraph/msgraph-sdk-java/issues/2026#issue-2330581773

[Benz1993com]

Ok