exo-explore
feature: headscale support
not sure how useful this is given network latency performance reduction, but i already implemented it, so here we are
not sure how useful this is given network latency performance reduction, but i already implemented it, so here we are
I agree about network latency, however this will be useful for exo v2 where we have distributed algo's that work better over slow connections with high latency.
Have you tested this with your own headscale server?
didn't think work can be distributed in ways that scales well without nanosecond level latency between all hidden layers, but i'm not very familiar with modern nnet architectures either;
yeah, seemed to work fine over headscale, tested with a mbp and 2xnvidia gpus, all in different networks;
at times when exo is started it produces strange output, sometimes complete gibberish, other times repeating words, but that manifested when testing with two nodes over local udp too