kv-cache topic

A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群

HDT3213

Completion After Prompt Probability. Make your LLM make a choice

kddubey

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

FMInference

Notes about LLaMA 2 model

hkproj

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

DRSY

This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture...

aju22

Supercharge Your LLM with the Fastest KV Cache Layer

LMCache

A high-performance RDMA distributed file system for fast LLM Inference and GPU Training

blackbird-io

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

therealoliver

Unified KV Cache Compression Methods for Auto-Regressive Models

Zefan-Cai