SQ

icon indicating a website link https://www.comp.nus.edu.sg/~s-zhang/

icon company SoC@NUS icon location Singapore icon location Research Fellow in SoC@NUS

Results 4 issues of SQ

Question about overlapped serving blocks

Consider a case that a pre-trained model is only hosted on three servers: the first one hosts blocks 1-4, the second hosts blocks 2-64, and the third hosts blocks 32-128....

ValueError: extra_stop_sequences require stop_sequence length to be exactly 1 token

Hi @borzunov, I started a private swarm with two GPU servers and built this web chat on another CPU node. When I input something, it raises errors like this `Oct...

Compute the FLOPs for each transformer layer of an LLM

3 comment

Hi @MrYxJ , Thanks for open-sourcing this excellent repo. May I ask what I should do if I only want to compute the FLOPs of a transformer layer in a...

请问该插件现在是否还有效？

RT