mlc-llm icon indicating copy to clipboard operation
mlc-llm copied to clipboard

[Feature Request] get only the last generated part of a message

Open tobrun opened this issue 2 years ago • 0 comments

🚀 Feature

Allow to get only the new parts of a message

Motivation

I'm trying to stream data and currently backend.getMessage returns the full message:

    fun getMessage(): String {
        assert(!Looper.getMainLooper().isCurrentThread)
        return get_message_!!.invoke().asString()
    }

tobrun avatar Jun 07 '23 14:06 tobrun