mlc-ai
[Feature Request] Decouple the Android ChatApp into library and app module
To make this project more portable and make it easier to integrate LLMs inside applications. It would be great to introduce a library module that contains the minimum code to integrate a LLM in an application. This would allow developers to build LLM models in this project, generate this library module in an aar format and integrate that as a binary into their applications.
In current state, you need to wrangle and untangle a lot of app code specific parts and makes it almost impossible to cleanly get the project migrated to an app codebase.
At the same time, this will also force us to think about the public API that such a LLM library would expose and forces us to decouple view element (which we reside in the example app) versus the model and presentation logic. There is even the potential of publishing prebuild library modules.
I'm happy to contribute these changes, we potentially want to be doing something similar for iOS.
This is a great proposal and is the direction we are working towards. I'm not particularly familiar with Android, so am CC'ing some related devs @spectrometerHBH @cyx-6 @tqchen
@tobrun great point.
The main thing we need to sort out is how to decouple native dependency as in many cases we need to package native libraries using different set of toolchains(e.g. cmake).
Here is how it can work, likely it will involve the following modules:
JVM components:
- M0:
ai.mlc.chatwhich is the ChatModule component that can be used as a java API - M1:
tvm4j-core, which is the core library of tvm4j runtime(that mlc chat uses) that depends on a JNI library(tvm4j_runtime_packed.so), note that the native components need to be packaged separately mainly so that consumers can take the models they would like to package.
Native component
- M2:
tvm4j_runtime_packed.so: the package JNI binary that we would like to link, we could try to make that as part of CMakeList, note that this is a place that different developers might benefit being able to directly built some of it from source, and integrate it into their project.- This is the native part, different ppl might have different target in mind, so a starting pt would be to provide a cmake list that enables people to build by setting the right target
So M0 and M1 are pure java component, which can be build and maintained separately. The way a new android or java project works is by:
- Build a project that depends on M0 and M1 (jar dependency)
- Add a binary build(CMake) that leverages existing cmake in M2 plus some of the path to model lib file(which we can provide helper script on), produces the
tvm4j_runtime_packed.sothat can be included in the app.
During start the app will look for the tvm4j_runtime_packed.so and then it should be a reasonably smooth sail
related note, iOS decoupling is done here https://github.com/mlc-ai/mlc-llm/pull/415
