state-of-open-source-ai
state-of-open-source-ai copied to clipboard
Published
20 hours ago •
premAI-io
premAI-io
Inference Optimization Chapter
Type
new chapter
Chapter/Page
Something else
Description
Doing training or inference models are fairly easy, when we have smaller number of parameters. But when the scale of parameters and data increases, it becomes increasingly difficult to optimize that (interms of compute and performance). So a dedicated chapter on inference optimization and arithmetics on resource calculations becomes very useful.
Some reference links:
- Mastering LLM Inference by NVIDIA
- Transformer Math 101
- Transformer Inference Arithmetic
- A twitter thread on some optimizations
