pytorch
please support normal java and scala3
🚀 The feature, motivation and pitch
Hi ,now we have javacpp-pytorch and storch and pytorch_on_scala3 on bytedeco org. please support us to load llm model on java and scala3 to generate token.. thanks
https://github.com/bytedeco/pytorch https://github.com/bytedeco/javacpp-presets/tree/master/pytorch https://github.com/bytedeco/pytorch_on_scala3_lesson
Alternatives
No response
Additional context
No response
RFC (Optional)
No response
Hi @mullerhai, could you expand on this request? I'm not too familiar with bytedeco.
please support us to load llm model on java and scala3 to generate token
Currently, we provide Java/Kotlin APIs to load LLMs and generate tokens. Is that what you are looking for?
Documentation: https://docs.pytorch.org/executorch/stable/using-executorch-android.html Code: https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch
Hi @mullerhai, could you expand on this request? I'm not too familiar with bytedeco.
please support us to load llm model on java and scala3 to generate token
Currently, we provide Java/Kotlin APIs to load LLMs and generate tokens. Is that what you are looking for?
Documentation: https://docs.pytorch.org/executorch/stable/using-executorch-android.html Code: https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch
I have seen the Java code of your project, but your Java code project is specifically for the Android mobile platform. The Android and Kotlin libraries it depends on will cause errors and cannot be used in ordinary Java projects and Scala 3 ordinary projects. This is because packages like androidx libs.core.ktx cannot be directly downloaded via Maven and SBT, which will result in errors. Your packaging method that only targets the Android platform is very unfavorable for other platforms to use executetorch. Please must use an executetorch package that does not include packages like androidx libs.core.ktx and is compatible with multiple platforms. We need an executetorch-java library that can be used normally on Linux, Mac, Windows PC, ARM, and x64. Please repackage and create an executetorch Java SDK without dependencies on Android platform libraries.
please repackage， do not include Androidx core library
Hi @mullerhai, could you expand on this request? I'm not too familiar with bytedeco.
please support us to load llm model on java and scala3 to generate token
Currently, we provide Java/Kotlin APIs to load LLMs and generate tokens. Is that what you are looking for?
Documentation: https://docs.pytorch.org/executorch/stable/using-executorch-android.html Code: https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch
today I have make sample code for this question ，please check it repo link -> https://github.com/mullerhai/torch-execute.git I imgrate the java code remove the andriod sdk dependency and write the same scala3 code and the resource directory have the exectorch jni library ? , the *.aar file is the jar ?
Hi @mullerhai, could you expand on this request? I'm not too familiar with bytedeco.
please support us to load llm model on java and scala3 to generate token
Currently, we provide Java/Kotlin APIs to load LLMs and generate tokens. Is that what you are looking for? Documentation: https://docs.pytorch.org/executorch/stable/using-executorch-android.html Code: https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch
today I have make sample code for this question ，please check it repo link -> https://github.com/mullerhai/torch-execute.git I imgrate the java code remove the andriod sdk dependency and write the same scala3 code and the resource directory have the exectorch jni library ? , the *.aar file is the jar ?
I have unzip the executorch-android.aar , it have libexecutorch.so for arm64-v8a and x86_64 , yet , only for android , we need executorch to support windows and linux and mac platform ,please develop these , you need repackage and recompile to generate all windows linux mac platform executorch sdk library
Hi @mullerhai, could you expand on this request? I'm not too familiar with bytedeco.
please support us to load llm model on java and scala3 to generate token
Currently, we provide Java/Kotlin APIs to load LLMs and generate tokens. Is that what you are looking for?
Documentation: https://docs.pytorch.org/executorch/stable/using-executorch-android.html Code: https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch
@lucylq @amyreese @per @ezyang If my words have offended you, please forgive me. This matter is extremely urgent — currently, Executorch can only be used on Android mobile devices, and we hope it will be compatible with all computer server-side platforms on java and scala3 like linux windows mac . I believe this is a problem we can resolve, so we ask you to make an attempt. We also aim to better promote Executorch to all users and platforms. Thank you.
Hi @mullerhai sorry for the late response 😅
. We need an executetorch-java library that can be used normally on Linux, Mac, Windows PC, ARM, and x64.
To help us better understand, could you share a bit more about your use case? What specific usecases or integrations would Java/Scala bindings enable for you?
Thank you
Hi @mullerhai sorry for the late response 😅
. We need an executetorch-java library that can be used normally on Linux, Mac, Windows PC, ARM, and x64.
To help us better understand, could you share a bit more about your use case? What specific usecases or integrations would Java/Scala bindings enable for you?
Thank you
do you know langchain4j and spring ai ?,now we use it to build llm rag application on java and scala3 , now you could see my llm usecase tutorial repo, https://github.com/mullerhai/langchain4j-scala-tutorial/ ,these so many use case that not run on andriod ,just running on windows and linux and mac. now we also use pytorch_on_scala3 to build llm model and training it. to build ai service and chat model ,we need executorch ,please regenerate and recompile the executorch source code ,and the java sdk split to part ,one for android , another part just remove the android core ktx ... jar dependencies for java and scala3 could use . firstly ,I think just java normal binding is ok , I could donate my scala3 code for executorch. if we make executorch running on java and scala normal platform ,we will fix up the gap about llm from python to java/scala. thanks
Thanks for the additional context about LangChain4j and Spring AI. What are the specific ecosystem use cases that we would unlock here with this support? Is it primarily the enterprise Java/Scala LLM ecosystem (RAG applications, AI services). Are there any other players waiting for such a support (Trying to gauge broader community demand) ? It looks like this could bridge Python→Java/Scala ML ecosystem gap.
In the shorter term, we could use your existing work as a foundation and leverage JavaCPP for cross-platform native library packaging.
But in the long term, as part of our ongoing ExecuTorch Android roadmap for '26, we're considering moving our Android/Java implementation to Kotlin. Given your use cases, Kotlin Multiplatform (KMP) could be a nice fit here. With KMP we can have shared common code (70-80%) for core logic/ modules — code that runs unchanged on Android and JVM. JavaCPP should work with Kotlin as well.
This way we can avoid maintaining forked code across Android and JVM. Over time, as Android continues to evolve there could be maintenance/ publishing overhead with separate codebases. What are your thoughts on this approach?
Moreover, would you/ your team like to contribute & collaborate with us on this project (both short & long term work)?
Thanks for the additional context about LangChain4j and Spring AI. What are the specific ecosystem use cases that we would unlock here with this support? Is it primarily the enterprise Java/Scala LLM ecosystem (RAG applications, AI services). Are there any other players waiting for such a support (Trying to gauge broader community demand) ? It looks like this could bridge Python→Java/Scala ML ecosystem gap.
In the shorter term, we could use your existing work as a foundation and leverage JavaCPP for cross-platform native library packaging.
But in the long term, as part of our ongoing ExecuTorch Android roadmap for '26, we're considering moving our Android/Java implementation to Kotlin. Given your use cases, Kotlin Multiplatform (KMP) could be a nice fit here. With KMP we can have shared common code (70-80%) for core logic/ modules — code that runs unchanged on Android and JVM. JavaCPP should work with Kotlin as well.
This way we can avoid maintaining forked code across Android and JVM. Over time, as Android continues to evolve there could be maintenance/ publishing overhead with separate codebases. What are your thoughts on this approach?
Moreover, would you/ your team like to contribute & collaborate with us on this project (both short & long term work)?
In China and Japan and South Korea ,more and more Internet company just build llm model rag , they use java as backend server to deploy llm model with langchian4j and spring AI ,no one want to deploy llm model use kotlin or deploy on andriod mobile , the kotlin andrioid also connect and operate llm request with backend , if you want imgrate to kmp it not good choice , I think everyone opensource can not ignore huge java backend user and company ,if we could use andriod core ktx dependency without error dependency in java or scala3 backend ,we could agree recently state . anyway . the executorch need windows linux mac *.dll *.so library .these package we should recompile and gerenete them . by the way most java/scala people unwilling to use kotilin and unknow how to download android core ktxt and download android core ktx always meet error in maven and sbt and mill package managerment .please do not make it complex , we do not need use javacpp , just only one easy way to try remove the android core ktx and regenerate the executorch *.dll *.so windows linux mac library. thanks.
The task is to decouple JNI and Java code from android specific build environment.
Hi @mullerhai as a workaround, can you popen a native executor_runner process in java?
The task is to decouple JNI and Java code from android specific build environment.
yes ， you are right。 just make them separate
The task is to decouple JNI and Java code from android specific build environment.
yes ， you are right。 just make them separate
I think maybe use facebook jni jar api could load the executorch *dll and *.so library is ok. now I think most important just to recompile and generate the executorch *.dll *.so library and package it in jar and use java native code invoke them.how do you think?
hi，@lucylq @amyreese @per @ezyang ，do we have some difficult problem to recompile and regenerate the executorch *dll *.so library file for linux windows and mac ? we need your help to recompile them
hi，@lucylq @amyreese @per @ezyang ，could you consider to implement executorch in linux with pure java ,we are eagerly to use , we just try once is ok , just could use it