fp8 topic

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization i...

NVIDIA

Microsoft Automatic Mixed Precision Library

Azure

An innovative library for efficient LLM inference via low-bit quantization

intel

Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.

aredden