fast-inference topic

Learning Efficient Convolutional Networks through Network Slimming, In ICCV 2017.

foolwood

[NeurIPS'23] Speculative Decoding with Big Little Decoder

kssteven418

This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"

dvlab-research

Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.

aredden

Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.

romsto