pentium3

icon indicating a website link https://pentium3.github.io/ icon indicating an email [email protected]

icon location Boston icon location emmmmmmmmmm

Results 104 issues of pentium3

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

https://github.com/SJTU-IPADS/PowerInfer

llm

LLM in a flash: Efficient Large Language Model Inference with Limited Memory

https://arxiv.org/pdf/2312.11514.pdf

llm

HexGen: Generative Inference of Foundation Model over Heterogeneous Decentralized Environment

1 comment

https://github.com/Relaxed-System-Lab/HexGen

llm

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache

https://arxiv.org/pdf/2401.02669.pdf

llm

Clipper: A Low-Latency Online Prediction Serving System

https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/crankshaw

EdgeServe: A Streaming System for Decentralized Model Serving

https://arxiv.org/pdf/2303.08028.pdf

Auto-WLM: machine learning enhanced workload management in Amazon Redshift

https://assets.amazon.science/5a/9d/338478254f9a9dde672fa84da2b7/auto-wlm-machine-learning-enhanced-workload-management-in-amazon-redshif.pdf

LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers

1 comment

https://arxiv.org/pdf/2310.03294.pdf

llm
SysforML

Efficient Streaming Language Models with Attention Sinks

https://github.com/mit-han-lab/streaming-llm

stream processing
llm
SysforML

Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM

https://arxiv.org/pdf/2104.04473.pdf

llm
SysforML