exo
exo copied to clipboard
Emit statistics
- tokens / sec
- memory usage
- gpu utilisation
- bytes sent / received
- num errors
- MFU (great metric. see e.g. https://x.com/tinygrad/status/1814519105346810038)
Should be easy to add this to opaque status, since that can hold arbitrary values and already gets broadcast to all nodes.