How to directly access an uploaded multipart files tmp file via path or FileInputStream

[nfode]

Ktor Version

1.1.3

Ktor Engine Used(client or server and name)

Server Core

JVM Version, Operating System and Relevant Context

JDK-8, Linux

Feedback

I upload large files as multipart/form-data and send them to Minio.

My problem is now that Minio needs the size of the to be uploaded data. Using part.streamProvider().available() is 0. So I searched the Ktor source code and found that uploaded files are uploaded and stored as tmp files and then an InputStream for them is provided through part.streamProvider().

Now my question is, if there is the possibility to get the file path or FileInputStream directly or if there is the possibility to pass the uploaded file as stream without storing it temporarily.

I looked into ContentNegotiation but I hesitate to do an own implementation for multipart/form-data since I am quite new to Kotlin.

Thanks in advance.

[cy6erGn0m]

Isn't the size available in the part headers? part.headers[HttpHeaders.ContentLength]?

[nfode]

This does not work because it is null. I tried it with: call.request.headers[HttpHeaders.ContentLength]

But then I get the following error:

java.io.EOFException: null
	at okio.RealBufferedSink.write(RealBufferedSink.java:107)
	at io.minio.HttpRequestBody.writeTo(HttpRequestBody.java:74)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:62)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
	at okhttp3.RealCall.execute(RealCall.java:69)
	at io.minio.MinioClient.executeReq(MinioClient.java:1097)
	at io.minio.MinioClient.execute(MinioClient.java:1056)
	at io.minio.MinioClient.executePut(MinioClient.java:1399)
	at io.minio.MinioClient.executePut(MinioClient.java:1421)
	at io.minio.MinioClient.putObject(MinioClient.java:3994)
	at io.minio.MinioClient.putObject(MinioClient.java:4086)
	at io.minio.MinioClient.putObject(MinioClient.java:3578)
	at handler.UploadHandler$Companion.uploadFiles(UploadHandler.kt:87)
	at handler.UploadHandler$Companion$uploadFiles$1.invokeSuspend(UploadHandler.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

The problem seems to be that the HttpHeaders.ContentLength outputs 570866810. But if I investigate the tmp file size with wc -c < filename it outputs 570866568.

I do not much about Streams but the EOFException comes from this mismatch?

Thanks for helping me!

[nfode]

@cy6erGn0m is this a bug or am I misunderstanding something?

[oleg-larshin]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.

[Stexxe]

As a workaround you can get the size of an uploaded file by reading all bytes from an underlying stream:

if (part is PartData.FileItem) {
    val input = part.provider()

    val arr = withContext(Dispatchers.IO) {
        input.asStream().readAllBytes()
    }
    println(arr.size)
}

[Jason-wam]

How to delete the temp file in 'AppData\Local\Temp' after upload?

[Stexxe]

It should be deleted automatically.

[Jason-wam]

It should be deleted automatically.

Thanks. I forgot to call 'part.dispose.invoke()' .