Add disk offloading for onsite LLM inference

[VictorOdede]

Enable large models that can't fit on the GPU to run inference by passing params back and forth between RAM and GPU-RAM

[mmirman]

This is what VLLM does: https://github.com/vllm-project/vllm