Siyuan Feng
Siyuan Feng
Hi, @areusch Thank you, for posting the analysis of the benefits and drawbacks of merging a module. I would like to point out that there are a few critical pieces...
@mbaret > I don't think it's fair or accurate to dismiss legitimate concerns of community contributors as 'subjective'. @areusch has already enumerated in some detail an 'objective' list of impacts...
The WIP PR: https://github.com/mlc-ai/mlc-llm/pull/142
finished at #142
`encode: 39.5 tok/s, decode: 26.0 tok/s` on Windows 11 with RTX-3080 `encode: 32.5 tok/s, decode: 17.1 tok/s` on Macbook Pro with M1Pro (16 GPUs) and macOS Ventura 13.3.1
`IfThenElseNode` is a stmt with two sub-stmt `then_case` and `else_case`, it equals to ```python if condition: then_case else: else_case ``` `tir.Select` is an expr with `true_value` and `false_value`, which equals...
encoder-decoder models like bert cannot use KVCache, which is definitely out of our scope. If you want to run bert model, TVM is a good choice instead of MLC-LLM
Hi all, as suggested in the thread, we held this thread for a while. And now it can be a good time to come back. Let me summarize the previous...
I withdraw this proposal in favor of the simpler and better process in #102
I totally agree it is useful. It can be done when the new parser is ready :)