Upload Large File via LargeFileUploadTask not working

[ihudedi]

Hi, I use your example and it's failed /Initialize the file input stream and get the file size InputStream file = new FileInputStream("File-Path"); long fileSize = file.available();

// Set the DriveItemUploadableProperties // This is used to populate the request to create an upload session DriveItemUploadableProperties driveItemUploadableProperties = new DriveItemUploadableProperties(); driveItemUploadableProperties.setName("fileName"); driveItemUploadableProperties.setFileSize(fileSize); Map<String, Object> additionalData = new HashMap<>(); additionalData.put("@microsoft.graph.conflictBehavior", "replace"); driveItemUploadableProperties.setAdditionalData(additionalData);

// Finish setting up the request body CreateUploadSessionPostRequestBody uploadSessionPostRequestBody = new CreateUploadSessionPostRequestBody(); uploadSessionPostRequestBody.setItem(driveItemUploadableProperties);

// Create the upload session String myDriveId = graphClient.me().drive().get().getId(); UploadSession uploadSession = graphClient.drives() .byDriveId(myDriveId) .items() .byDriveItemId("root/fileName") .createUploadSession().post(uploadSessionPostRequestBody); // Create the large file upload task LargeFileUploadTask<DriveItemUploadableProperties> uploadTask = new LargeFileUploadTask(graphClient.getRequestAdapter(), uploadSession, file, fileSize, DriveItemUploadableProperties::createFromDiscriminatorValue); try{ UploadResult<DriveItemUploadableProperties> uploadResult = uploadTask.upload(); } catch(ApiException | InterruptedException exception) { System.out.println(exception.getMessage());

} I am failing when trying to create UploadSessin I use in byDriveItemId also your example ("root/fileName" and it's not working Thanks, Itay

[araneolus]

The Mistake is at this point .byDriveItemId("root/fileName") Try .byDriveItemId("root:/fileName:") Then you will avoid this cliff, but you will not get any further with upload, because it always returns a ReturnCode 400 with the message "com.microsoft.kiota.ApiException: generalException". I have tried a lot, e.g. omitting the name and size in DriveItemUploadableProperties, but it didn't help. The first chunk is uploaded to the server and then the error code 400 appears.

[araneolus]

I always get the return code 400 when uploading. I have tried it with different DriveItemUploadableProperties and maxslicesice of 327680 but unfortunately without success. DriveItemUploadableProperties:

{
	"item": {
		"@microsoft.graph.conflictBehavior": "replace"
	}
}

or

{
	"item": {
		"@microsoft.graph.conflictBehavior": "replace"
	},
	"deferCommit": true
}

or

{
	"item":{
		"description":"test",
		"fileSize":8589114,
		"name":"test.log",
		"@microsoft.graph.conflictBehavior":"replace"
	}
}

If I use the method to upload small files, it works well

RequestInformation requestInformation = graphClient.drives().byDriveId(drive.getId()).items()
.byDriveItemId("root:/" + destination + ":").content().toPutRequestInformation(inputstream);
URI uriIncludesConflictBehavior = new URI(
requestInformation.getUri().toString() + "[email protected]=rename");
requestInformation.setUri(uriIncludesConflictBehavior);
graphClient.getRequestAdapter().sendPrimitive(requestInformation, null, InputStream.class);

[ihudedi]

Hi @araneolus Also with byDriveItemId("root:/fileName:") not wokring Thanks, Itay

[araneolus]

Which error do you got ? Item already exists, or a DateTimeException ?

[ihudedi]

Hi @araneolus Always datetime error

[araneolus]

You have to use the com.microsoft.kiota Version 1.0.2. The cause you can find at https://github.com/microsoftgraph/msgraph-sdk-java/issues/1785

[araneolus]

At the console I see following Debug Message, before I receive the 400 error code

14 Feb. 2024 13:38:33.573- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 16384 DATA          
14 Feb. 2024 13:38:33.599- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 16384 DATA          
14 Feb. 2024 13:38:33.625- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 16384 DATA          
14 Feb. 2024 13:38:33.651- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003  3898 DATA          END_STREAM
14 Feb. 2024 13:38:33.656- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003     4 WINDOW_UPDATE 
14 Feb. 2024 13:38:33.656- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000000     4 WINDOW_UPDATE 
14 Feb. 2024 13:38:33.661- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003  1149 HEADERS       END_HEADERS
14 Feb. 2024 13:38:33.663- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003   174 DATA          
14 Feb. 2024 13:38:33.663- Http2 [OkHttpxx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003     0 DATA          END_STREAM

[ihudedi]

Hi @araneolus Now I am getting Exception in thread "main" com.microsoft.graph.models.odataerrors.ODataError: Invalid request at com.microsoft.graph.models.odataerrors.ODataError.createFromDiscriminatorValue(ODataError.java:36) at com.microsoft.kiota.serialization.JsonParseNode.getObjectValue(JsonParseNode.java:210) at com.microsoft.kiota.http.OkHttpRequestAdapter.lambda$throwIfFailedResponse$0(OkHttpRequestAdapter.java:672) at com.microsoft.kiota.ApiExceptionBuilder.(ApiExceptionBuilder.java:26) at com.microsoft.kiota.http.OkHttpRequestAdapter.throwIfFailedResponse(OkHttpRequestAdapter.java:671) at com.microsoft.kiota.http.OkHttpRequestAdapter.send(OkHttpRequestAdapter.java:279) at com.microsoft.graph.drives.item.items.item.createuploadsession.CreateUploadSessionRequestBuilder.post(CreateUploadSessionRequestBuilder.java:60) at com.microsoft.graph.drives.item.items.item.createuploadsession.CreateUploadSessionRequestBuilder.post(CreateUploadSessionRequestBuilder.java:45) at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:116)

[ihudedi]

Hi @araneolus I added the updated graph core to 3.1.2 and now I am getting new error: Exception in thread "main" com.microsoft.kiota.ApiException: generalException at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45) at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:61) at com.microsoft.graph.core.requests.upload.UploadSliceRequestBuilder.put(UploadSliceRequestBuilder.java:69) at com.microsoft.graph.core.tasks.LargeFileUploadTask.uploadSlice(LargeFileUploadTask.java:207) at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:131) at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111) at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:138)

[araneolus]

You are now as far along as I am. Welcome to the club :-)

[araneolus]

This is the structure of the 400 Response url=https://xx-my.sharepoint.com/personal/hugo_musterman_xxx_de/_api/v2.0/drives/.../items/.../uploadSession?guid=%27...%27&overwrite=True&rename=False&dc=0&tempauth=...}

[ihudedi]

Hi @Ndiritu same issue for version 6.2.0 Thanks, Itay

[ramsessanchez]

Hi @ihudedi , I have put together a pr to patch the largeFileUpload, hopefully we cant get this in by next minor version release, 6.3.0, apologies for the hassle and thank you for using the SDK.

[ramsessanchez]

https://github.com/microsoft/kiota-java/pull/1088

[ihudedi]

Hi @ramsessanchez Still same issue and not working in version 6.3.0 Exception in thread "main" com.microsoft.kiota.ApiException: generalException at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45) at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62) at com.microsoft.graph.core.requests.upload.UploadSliceRequestBuilder.put(UploadSliceRequestBuilder.java:69) at com.microsoft.graph.core.tasks.LargeFileUploadTask.uploadSlice(LargeFileUploadTask.java:207) at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:131) at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111) at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:201) Thanks, Itay

[ihudedi]

Hi @ramsessanchez Is there any updates? Thanks, Itay

[ffcdf]

Hi @ihudedi.

With the new version of microsoft-graph 6.4.0 and microsoft-kiota 1.0.5, the first bytes of the file are loaded, but then I receive the error:

java.lang.IndexOutOfBoundsException at java.base/java.io.FileInputStream.readBytes(Native Method) at java.base/java.io.FileInputStream.read(FileInputStream.java:276) at com.microsoft.graph.core.tasks.LargeFileUploadTask.chunkInputStream(LargeFileUploadTask.java:280)

No success here yet :(...

Any successes out there?

[ihudedi]

Hi @ffcdf It was working fine in v 5.80. What is broken in this version?

[ramsessanchez]

@ihudedi , we have updated the kiota-java dependencies in graph-core. Please expect to see these changes reflected on the next version of the sdk, or manually update your project to use graph-core 3.1.6. Follow up if issues persist.

[ffcdf]

Hi @ihudedi and @ramsessanchez

I'm still a beginner in Java, but I'm testing uploading files using the LargeFileUploadTask class from microsoft-graph-core 3.1.6.

However, I get the java.lang.IndexOutOfBoundsException error and the upload doesn't work. The error occurs during the second pass in the chunkInputStream method because stream.read tries to load the read bytes to a position that does not exist in the buffer.

I downloaded the source code and made some adjustments here. With the adjustments, the file is uploaded to OneDrive, but at the end of the process it still throws an exception that I was unable to resolve:

generalException com.microsoft.kiota.ApiException: generalException at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45) at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62) at com.microsoft.graph.core.requests.upload.UploadSessionRequestBuilder.get(UploadSessionRequestBuilder.java:52) at com.microsoft.graph.core.tasks.LargeFileUploadTask.updateSessionStatus(LargeFileUploadTask.java:206)...

The adjustments I made were: private byte[] chunkInputStream(FileInputStream stream, int begin, int length) throws IOException { //byte[] buffer = new byte[length]; //int lengthAssert = stream.read(buffer, begin, length); ByteBuffer byteBuffer = ByteBuffer.allocate(length); int lengthAssert = stream.getChannel().read(byteBuffer); byteBuffer.flip(); byte[] buffer = byteBuffer.array(); byteBuffer.clear(); assert lengthAssert == length; return buffer; }

Still trying to find a solution without exceptions....

[ihudedi]

Hi @ramsessanchez I am encounter same issue as @ffcdf Second chunk is failed in version core 3.1.6 with graph api 6.4.0 Thanks, Itay

[abhishekmohite]

I am running into the same issue when uploading large files using code from here https://learn.microsoft.com/en-us/graph/sdks/large-file-upload?tabs=java & https://github.com/microsoftgraph/msgraph-sdk-java/blob/dev/docs/upgrade-to-v6.md#large-file-upload-enhancements

I am using microsoft-graph 6.4.0 and kiota 1.0.6.

[ihudedi]

Hi @ramsessanchez Any updates regarding this issue? I am waiting for a fix more than 3 weeks. It was working fine in v5.8 Thanks, Itay

[baywet]

Hi everyone, Thanks for your patience on the matter. Ramses and I had a prolonged discussion on that topic yesterday after his analysis of the problem. As expected, some of the confusion lies in the fact that difference services (Exchange, OneDrive for Business/SharePoint, OneDrive for consumers) behave differently.

The upload seems to be working in most cases, while some services reply with 200/201 and a response body (which is what the current implementation handles properly), others reply with 201 and no response body. The later case derails the code here, where okhttp throws when attempting to read an inexistent body (rightfully so). https://github.com/microsoftgraph/msgraph-sdk-java-core/blob/94093eaa875c0f4088e0be6457358b51638d379f/src/main/java/com/microsoft/graph/core/requests/upload/UploadResponseHandler.java#L54

The fix will be to check for a content-length header prior to reading the body, and return early if none is present/value is zero.

In addition to that, we'll work to triple check the implementation matches the dotnet implementation's behaviour since it's been in production for over a year now, and the adoption is much larger than java (exposing it to more potential use cases).

Thank you everyone for your patience while we take the time to get it right for good, and sorry again about the inconvenience caused.

[andreidore]

Hi

In my case, it returns 400 ( Bad request ) with this example: https://learn.microsoft.com/en-us/graph/sdks/large-file-upload?tabs=java

It is very frustrating because the HTTP responds only with status code 400. No message, nothing. I wonder why it is a bad request.

[ihudedi]

Hi @baywet Could you please let us know when fix will be release? Thanks, Itay

[rolansh]

@baywet Is this relevant to a situation when in UploadResponseHandler.handlerResponse at line 60 response.isSuccessful() is false due to response.code == 404? Or does it indicate another issue?

https://github.com/microsoftgraph/msgraph-sdk-java-core/blob/94093eaa875c0f4088e0be6457358b51638d379f/src/main/java/com/microsoft/graph/core/requests/upload/UploadResponseHandler.java#L60

[ihudedi]

Hi @baywet @Ndiritu @ramsessanchez Any updates on this issue? I am waiting for a fix for more than a month. This functionality was working well on v5. Thanks, Itay

[baywet]

@ihudedi I want to be clear regarding timelines and expectations here: The help provided on GitHub comes with no SLA whatsoever, it is best effort taking other priorities and resourcing into account. Pinging us on multiple issues just to ask for timelines doesn't help getting to a faster resolution, to the contrary, it creates more noise to sort through on our end. If you are time constrained, the ONLY way to get support with SLAs is through opening a support ticket (Azure, Microsoft 365). That said, we are working on a fix, you can follow the progress here https://github.com/microsoft/kiota-java/pull/1119

[araneolus]

This morning I have tried the Version 1.1.1 of kiota but I got still an error by upload huge files. I have tried both ByteArrayInputStream and Files.newInputStream as InputStream. Maybe I was too optimistic and thought that solving the microsoft/kiota-java#1119 ticket would solve the problem. If I can contribute anything to the solution please tell me what I should do.

21 März 2024 09:21:31.303- Http2 [OkHttp alfamediaroedermark-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000000     4 WINDOW_UPDATE 
21 März 2024 09:21:31.736- Http2 [OkHttp alfamediaroedermark-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003  1167 HEADERS       END_HEADERS
21 März 2024 09:21:31.738- Http2 [OkHttp alfamediaroedermark-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003   228 DATA          
21 März 2024 09:21:31.755- Http2 [OkHttp alfamediaroedermark-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003     0 DATA          END_STREAM
21 März 2024 09:21:31.755- TaskRunner [] DEBUG okhttp3.internal.concurrent.TaskLoggerKt 60 log - Q10000 scheduled after   0 µs: OkHttp ConnectionPool
21 März 2024 09:21:31.755- TaskRunner [OkHttp TaskRunner] DEBUG okhttp3.internal.concurrent.TaskLoggerKt 60 log - Q10000 starting              : OkHttp ConnectionPool
21 März 2024 09:21:31.756- TaskRunner [OkHttp ConnectionPool] DEBUG okhttp3.internal.concurrent.TaskLoggerKt 60 log - Q10000 run again after 299 s : OkHttp ConnectionPool
21 März 2024 09:21:31.756- TaskRunner [OkHttp TaskRunner] DEBUG okhttp3.internal.concurrent.TaskLoggerKt 60 log - Q10000 finished run in 789 µs: OkHttp ConnectionPool
21 März 2024 09:21:31.757- OneDriveObject [] DEBUG net.araneolus.vectorius.route.protocol.onedrive.OneDriveObject$1 1009 report - Uploaded 5242880 bytes of 50389889 total bytes
21 März 2024 09:21:31.759- OneDriveObject [] ERROR net.araneolus.vectorius.route.protocol.onedrive.OneDriveObject 1069 put - [put] 99facae8-bd1a-42f1-905a-c4e8d2927817 Can't write 'testfolder/8081.dmp' Range [5242880, 5242880 + 5242880) out of bounds for length 5242880
Can't write 'testfolder/8081.dmp' Range [5242880, 5242880 + 5242880) out of bounds for length 5242880
	at net.araneolus.vectorius.route.task.directories.directory.transfer.Common$4.call(Common.java:857)
	at net.araneolus.vectorius.route.task.directories.directory.transfer.Common$4.call(Common.java:1)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run(ThreadPerTaskExecutor.java:314)
	at java.base/java.lang.VirtualThread.run(VirtualThread.java:309)