msgraph-sdk-java icon indicating copy to clipboard operation
msgraph-sdk-java copied to clipboard

JVM proxy settings are ignored by LargeFileUploadTask

Open kekolab opened this issue 3 years ago • 3 comments

The JVM proxy seems to be ignored when sending calls to onedrive api through LargeFileUploadTask. I start the JVM with the typical proxy settings, i.e. with -Dhttp.proxyHost=..., and -Dhttps.proxyHost and so on. All the calls to the API are correctly executed, but when it comes to LargeFileUploadRequest.upload, an exception is thrown by the underlying okhttp client. I hereby copy and paste the graph log messages, that testify how all the previous requests are executed correctly and the stacktrace.

Expected behavior

LargeFileUploadRequest.upload should not ignore the JVM proxy settings

Actual behavior

LargeFileUploadRequest.upload ignores the JVM proxy settings

Steps to reproduce the behavior

The SDK version I am using is microsoft-graph-5.30.0

Log and stacktrace (behind the proxy)

Getting the root folder

12:34:40.559 DEBUG  - Starting to send request, URL https://graph.microsoft.com/v1.0/drive/root
12:34:40.569 DEBUG  - Request Method GET
12:34:41.346 DEBUG  - Response code 200, OK
12:34:41.349 DEBUG  - Response json
12:34:41.391 DEBUG  - Deserializing type DriveItem

Creating a folder in the root folder

12:34:42.549 DEBUG  - Starting to send request, URL https://graph.microsoft.com/v1.0/me/drive/items/83A259E7E5F58E69!106/children
12:34:42.550 DEBUG  - Request Method POST
12:34:42.551 DEBUG  - Sending com.microsoft.graph.models.DriveItem as request body
12:34:42.551 DEBUG  - Serializing type DriveItem
12:34:42.770 DEBUG  - Response code 201, Created
12:34:42.771 DEBUG  - Response json
12:34:42.777 DEBUG  - Deserializing type DriveItem

Creating the upload session

12:34:42.846 DEBUG  - Starting to send request, URL https://graph.microsoft.com/v1.0/me/drive/items/83A259E7E5F58E69!30351:/random7636145754369844697.tmp:/microsoft.graph.createUploadSession
12:34:42.847 DEBUG  - Request Method POST
12:34:42.847 DEBUG  - Sending com.microsoft.graph.models.DriveItemCreateUploadSessionParameterSet as request body
12:34:42.847 DEBUG  - Serializing type DriveItemCreateUploadSessionParameterSet
12:34:42.957 DEBUG  - Response code 200, OK
12:34:42.957 DEBUG  - Response json
12:34:42.961 DEBUG  - Deserializing type UploadSession

Sending the bytes through the upload session

12:34:42.980 DEBUG  - Starting to send request, URL https://api.onedrive.com/rup/83a259e7e5f58e69/eyJSZXNvdXJjZUlEIjoiODNBMjU5RTdFNUY1OEU2OSEzMDM1MSIsIlJlbGF0aW9uc2hpcE5hbWUiOiJyYW5kb203NjM2MTQ1NzU0MzY5ODQ0Njk3LnRtcCJ9/4mQDQjtHqCi32O6vtG0I02sbjicLKu8zTfVQnbdzIjNL9vp2dc6x6rfA56byfJmABQx-7AFwa2AIx2rn5tz6YEqLsGMO73Zl6RDwUl9YCTY20/eyJuYW1lIjoicmFuZG9tNzYzNjE0NTc1NDM2OTg0NDY5Ny50bXAiLCJAbmFtZS5jb25mbGljdEJlaGF2aW9yIjoicmVwbGFjZSJ9/4wUEY3dm3oczGBZORRLiWqgBWcftpPtwOyNjX6nKPg5nBYWKPVNOofmbWnJU1mYP_YJ0yD8uFSIewqRNS-vabmderxZvLphS1vLRB4UqfsUb26LYAN0rxEOXkr5FyAAI0KGq3gxLGtIXSeffNZ39BKDgmjj8RE5uTvoSJTwPymxLBoM6m0Y8pY9OUEvwwYl3tTJPRsBXYLGwrPZc0zmBGBKuFF0NIeN5nD9-8OeqGfzPT5IqVTrsQFjgjnfYSLmZCW2JGYQ2ZWSTNtdwcd9aAoqFYeGOoLiyPeyRya4CZ8jHVt26zwEW6lgNtGbwnDF9wkT1SLyw92qyBsASBa3eunaDn48PsAHXwL1eSv6svzwwzqcf0-T0LZdGtvYcBbNfyvdqzDZpvqPaLhCCr6AQ_GjdfIQNIknGolb00_TwuihihQcTENn0CNOapG5LK5yQHA3dEAzu-7t6blig8365iZUFRIPfXYlMW52wVYd-wYKYQNqFCJCJNzdkHL7cAvAzxQfJWbhTZGfHxJ3fDNwtKCq9of-9ym7TVvAATBg-AxNZ_0Bt0kIFlbu-x5gpYoproa
12:34:42.983 DEBUG  - Request Method PUT
12:34:42.984 DEBUG  - Sending byte[] as request body
com.microsoft.graph.core.ClientException: Request failed with error, retry if necessary.
	at com.microsoft.graph.tasks.LargeFileUploadRequest.upload(LargeFileUploadRequest.java:100)
	at com.microsoft.graph.tasks.LargeFileUploadTask.uploadAsync(LargeFileUploadTask.java:167)
	at com.microsoft.graph.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:255)
	at com.microsoft.graph.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:274)
	at com.microsoft.graph.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:286)
	at com.microsoft.graph.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:297)
	at kekolab.ksync.provider.onedrive.OnedriveFolder.upload(OnedriveFolder.java:53)
	at kekolab.ksync.provider.onedrive.OnedriveFolder.upload(OnedriveFolder.java:1)
	at kekolab.ksync.ItemTests.upload(ItemTests.java:59)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:93)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: com.microsoft.graph.core.ClientException: Error executing the request
	at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:410)
	at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:225)
	at com.microsoft.graph.tasks.LargeFileUploadRequest.upload(LargeFileUploadRequest.java:98)
	... 44 more
Caused by: java.io.IOException: Failed to authenticate with proxy
	at okhttp3.internal.connection.RealConnection.createTunnel(RealConnection.kt:476)
	at okhttp3.internal.connection.RealConnection.connectTunnel(RealConnection.kt:262)
	at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:201)
	at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
	at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
	at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
	at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at com.microsoft.graph.httpcore.RedirectHandler.intercept(RedirectHandler.java:137)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at com.microsoft.graph.httpcore.RetryHandler.intercept(RetryHandler.java:177)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at com.microsoft.graph.httpcore.AuthenticationHandler.intercept(AuthenticationHandler.java:57)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at com.microsoft.graph.httpcore.TelemetryHandler.intercept(TelemetryHandler.java:69)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
	at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:407)
	... 46 more

Log (without being behind a proxy)

13:21:27.203 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Starting to send request, URL https://graph.microsoft.com/v1.0/drive/root
13:21:27.240 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Request Method GET
13:21:28.152 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response code 200, OK
13:21:28.158 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response json
13:21:28.187 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Deserializing type DriveItem
13:21:29.564 INFO  [Microsoft Onedrive (personal) ([email protected]) <-> Local folder (C:\Users\03200692\mcloud)] - State correctly presisted for syncer with id 1
13:21:29.712 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Starting to send request, URL https://graph.microsoft.com/v1.0/me/drive/items/83A259E7E5F58E69!106/children
13:21:29.712 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Request Method POST
13:21:29.713 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Sending com.microsoft.graph.models.DriveItem as request body
13:21:29.713 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Serializing type DriveItem
13:21:29.981 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response code 201, Created
13:21:29.981 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response json
13:21:29.986 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Deserializing type DriveItem
13:21:30.056 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Starting to send request, URL https://graph.microsoft.com/v1.0/me/drive/items/83A259E7E5F58E69!30354:/random10406167154923963580.tmp:/microsoft.graph.createUploadSession
13:21:30.056 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Request Method POST
13:21:30.057 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Sending com.microsoft.graph.models.DriveItemCreateUploadSessionParameterSet as request body
13:21:30.057 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Serializing type DriveItemCreateUploadSessionParameterSet
13:21:30.617 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response code 200, OK
13:21:30.618 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response json
13:21:30.626 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Deserializing type UploadSession
13:21:30.646 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Starting to send request, URL https://api.onedrive.com/rup/83a259e7e5f58e69/eyJSZXNvdXJjZUlEIjoiODNBMjU5RTdFNUY1OEU2OSEzMDM1NCIsIlJlbGF0aW9uc2hpcE5hbWUiOiJyYW5kb20xMDQwNjE2NzE1NDkyMzk2MzU4MC50bXAifQ/4mDdOhshqUFKiSZsvuBydmckX8x-58uIv6400LN6rrTW5SURmTrKuvEJ1SDhjkczJowEtu9BkbaPBXXXM-4L8sLZ4FdWqBYSjiLP-RD2fZMh0/eyJuYW1lIjoicmFuZG9tMTA0MDYxNjcxNTQ5MjM5NjM1ODAudG1wIiwiQG5hbWUuY29uZmxpY3RCZWhhdmlvciI6InJlcGxhY2UifQ/4w_AlqgwQPHqgZK_S55HaAjM06nRkMpttaXLzq6EGw6lVa2prscuOQRrgnCFpBE7DNX0lquy2L2GU3CnGD-uat6S2ODn3vdRYP1N-G0vaiLH1xxSwwMQmRHNa94HvCfuOatH_Kjt6Tja9fWKkHUoE7yCnS-2Ek_MRPlTEYru33G7-KUiy5v_ohmb0orKBKWiDECUxFoF_6LqMH1L6QptezchrX5BdoAUp7VUdQzGh0wDaAtvcUlav5hulggDcLwscCdbDCUwOMHV7fQndB0kRzkHG6zvMvbGyxmSZy1YkL7RiY5XxtDASLb6i1q69a1ZsSEFuITytmjkNigYUhw-4MOyu2d5kabCB976ztBss8iHTT-rGG1FdXZkuhaSic5CIO6GwMQv31se0u22F8q1y_yn_POMpXoKHDDM3qqUuYifB0HpFjxkv6mzmoU9cTlNsktQE7VTDnQE11eYekCrnC2Q14NcfdLPatHl2ruhKH2Wc1ioTbnp3WPSOjklmv-h3tG-iRlAf4ZZE5jNJ3Kb0wxoNVRQ1XqZJVuEvU2SgxLiKAF84FnHDM6oVUtQXypX8p
13:21:30.649 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Request Method PUT
13:21:30.649 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Sending byte[] as request body
13:21:31.769 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Response code 201, 
13:21:31.769 DEBUG [Microsoft Onedrive (personal) ([email protected])] - StatefulResponse is handling the HTTP response.
13:21:31.774 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Deserializing type UploadSession
13:21:31.775 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Upload session is completed (ODSP), uploaded item returned.
13:21:31.777 DEBUG [Microsoft Onedrive (personal) ([email protected])] - Deserializing type DriveItem

The code

public void upload(IFile file) throws IOException {
  try (InputStream inputStream = file.getInputStream()) {
	  DriveItemCreateUploadSessionParameterSet uploadParams = DriveItemCreateUploadSessionParameterSet
			  .newBuilder()
			  .withItem(new DriveItemUploadableProperties())
			  .build();
  
	  UploadSession uploadSession = getGraphServiceClient()
			  .me()
			  .drive()
			  .items(getItem().id)
			  .itemWithPath(file.getName())
			  .createUploadSession(uploadParams)
			  .buildRequest()
			  .post();
  
	  LargeFileUploadTask<DriveItem> largeFileUploadTask = new LargeFileUploadTask<>(uploadSession,
			  getGraphServiceClient(), 
			  inputStream, 
			  file.getLength(), 
			  DriveItem.class);
  
	  DriveItem item = largeFileUploadTask.upload().responseBody;
  } catch (IOException e) {
        throw e;
  }
}

kekolab avatar Jul 19 '22 10:07 kekolab

Hi @kekolab, would you mind sharing the code in which you setup the proxy? It seems perhaps there is an authentication issue. Take a look at this documentation which provides some information on setting up the proxy for the client. https://docs.microsoft.com/en-us/graph/sdks/customize-client?tabs=java#configuring-the-http-proxy-for-the-client

ramsessanchez avatar Aug 08 '22 22:08 ramsessanchez

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

ghost avatar Aug 12 '22 23:08 ghost

@ramsessanchez , I pasted the code I use to upload the file above. This is how I create the client

protected OnedriveAccount(OnedriveState state) {
		this.state = state;
		this.credential = new Credential.Builder(BearerToken.authorizationHeaderAccessMethod())
				.setClientAuthentication(new ClientParametersAuthentication(OnedriveAccountBuilder.CLIENT_ID, null))
				.setJsonFactory(new GsonFactory())
				.setTokenServerEncodedUrl(OnedriveAccountBuilder.TOKEN_URL)
				.setTransport(new NetHttpTransport())
				.setRefreshListeners(Collections.singleton(this)).build();
		this.credential.setAccessToken(state.getAccessToken());
		this.credential.setExpirationTimeMilliseconds(state.getExpirationTime());
		this.credential.setRefreshToken(state.getRefreshToken());
		this.logger = LogManager.getLogger(String.format("[%s (%s)]", BUILDER.getName(), getUsername()));
		Log4jGraphLogger graphLogger = new Log4jGraphLogger(logger);
		this.graphServiceClient = new GraphServiceClient.Builder<>()
				.authenticationProvider(new TokenCredentialAuthProvider(new GoogleOAuth2TokenCredential(this.credential)))
				.logger(graphLogger)
				.buildClient();
		logger.debug("Account correctly created/restored");
	}

As you can see, I do not configure it as you pointed out, but I configure globally through java properties the proxy for all the JVM. What is weird is that any call the client makes works perfectly but this one.

kekolab avatar Aug 13 '22 13:08 kekolab

@ramsessanchez to verify if this is addressed in new version.

ddyett avatar May 25 '23 21:05 ddyett

Hi, Sorry for the late reply, we've been working hard to release a new major version of the SDK Since this is an issue with the older version, we'll be closing it for now. Please go ahead and try with this new version, if you're still facing issues, feel free to open a new issue.

baywet avatar Feb 08 '24 13:02 baywet