Sam Stoelinga

icon indicating a website link https://www.kubeai.org/

icon location San Francisco Bay Area icon location Creator of KubeAI, a K8s operator to serve LLMs in production.

Results 133 issues of Sam Stoelinga

Pathways: reuse setup_spmd for pathways init

2 comment

Improve pathways checkpoint load times

* Utilize a shared memory between the Jax client and pathways proxy for data heavy transfers e.g. device_puts. * Increase threads of ThreadPoolExecutor from 32 (python default) to 192. *...

WIP: pathways checkpointing experiments