LLM-VM

LLM-VM copied to clipboard

Implement FSDP for training large datasets

1

Definition of done: Implement training large models using FSDP to accelerate training on large datasets. Reference: https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/

TheRealVish

Python profiler for CPU, GPU and Memory

7

Could possibly be done using `Scalene`

VictorOdede

Add disk offloading for onsite LLM inference

1

Enable large models that can't fit on the GPU to run inference by passing params back and forth between RAM and GPU-RAM

VictorOdede

Load-balancing / auto-scaling for LLM serving on AWS

3

VictorOdede

Load-balancing / auto-scaling for LLM serving on Google Cloud

2

VictorOdede

Load-balancing / auto-scaling for LLM serving on Azure

5

VictorOdede

Hosted inference + finetuning interface for Azure

3

VictorOdede

LLM-VM does not support multiple GPUs currently

4

Currently LLM-VM does not support multiple GPU setups. Using runpod, I rented a setup with 2 RTX 3090 GPUs. Well running the local Bloom model example from the [docs](https://anarchy.ai/get_started/quickstart/completions). I...

MehmetMHY

Blog tickets

5

We discussed YouTube videos in our latest triage but before we publish any videos, it would be good to have the video content as blogs.

INF800

Mega Ticket - Gallery

6

open bounty for demo applications that work to add to a curated example gallery

cartazio