Foundation Model Inference

icon location Inference Systems for Foundation Models

Results 2 repositories owned by Foundation Model Inference

FlexGen

9.0k
Stars
527
Forks
83
Watchers

Running large language models on a single GPU for throughput-oriented scenarios.

verified publisher icon FMInference

H2O

295
Stars
25
Forks
Watchers

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

verified publisher icon FMInference