Ruihang Lai

icon indicating a website link https://ruihanglai.com icon indicating an email [email protected]

icon company Carnegie Mellon University icon location Pittsburgh, United States icon location Third-year PhD student at CMU / Undergrad from ACM Class, SJTU / ML Systems / Deep Learning Compilers / @apache TVM PMC

Results 136 comments of Ruihang Lai

Demo site generation error: Required limit

Hello @mindgitrwx and @ahmartello, thanks for the feedback. This issue is essentially because running the model needs to allocate larger buffers for intermediate values. At this moment we are running...

Loading models from disk

Hi @bakkot, thanks for your great suggestion. Loading from local files is not hard. In the future we can provide such an option for people to choose, on “whether to...

Demo fails on latest tvm/unity branch

Hi @infil00p, thanks for your feedback! Would you mind providing some more detailed information so that we can help check? We are using the mlc-ai/relax repository (https://github.com/mlc-ai/relax) as codebase for...

[Bug] gen_config doesn't add context_window_size to model_config every time

Thanks you so much for digging and reporting. We will look into and get a fix!

[Discuss][IR] Relax PrimValue

Thanks @YuchenJin for the proposal! I have a few comments on that. * With PrimValue, we now have two options for a Relax high-level operator API. Say we are designing...

[DISCUSS] Relax Pass Writing Paradigm

Here’s the note and examples I used today :-) Welcome to leave any comment or discuss 😆 !

package Base does not exist

@starlitsky2010 Thanks for the feedback! Indeed we missed some dependency steps before and that was not complete enough for building the app locally. We just pushed #124 which contains more...

[KVCache] Support passing in attn_score_scaling_factor into KV cache

Given https://github.com/flashinfer-ai/flashinfer/pull/126 has been merged, let's bump 3rdparty/flashinfer to the latest FlashInfer

[KVCache] Support passing in attn_score_scaling_factor into KV cache

Also there is a format issue https://ci.tlcpack.ai/blue/organizations/jenkins/tvm-lint/detail/PR-16606/3/pipeline

[Bug][Parsing] Mutual recursion fails to parse inside an if branch

We now seem to have the restriction that the body of function must be LeafExpr, IIRC. Probably we can improve it with a better error msg.