exo icon indicating copy to clipboard operation
exo copied to clipboard

Document the format of the manual configuration file and add --get-device-capabilities

Open kdkd opened this issue 8 months ago • 0 comments

This patch does two things to assist users that make use of the manual peer discovery system.

First, it adds a comment that documents the format of the config file that --discovery-config-path is expecting. As far as I can tell, nowhere in the source or documentation shows how this file is supposed to look.

Second, it adds a command-line parameter called --get-device-capabilities that outputs the current device's device_capabilities block. We needed this to be able to automate creating the manual configuration file, since those values aren't really easy to determine outside of exo. Example:

# exo --get-device-capabilities
{
  "model": "Mac Studio",
  "chip": "Apple M3 Ultra",
  "memory": 524288,
  "flops": {
    "fp32": 54.26,
    "fp16": 108.52,
    "int8": 217.04
  }
}

This is a little hairy because it needs to execute BEFORE anything else can output to the console, but the asyncio loop hasn't been started until after other things have been printed out and all the device capability code is async. So I added a small function to helpers.py to create a tiny event loop. There's probably a better way of refactoring this so all the real argument processing and startup happens in main() instead of at the top level, but that seemed too invasive for such a small feature. Maybe config_uvloop could execute earlier, but I'm not sure of the history for why that's pulled out separately.

kdkd avatar Apr 02 '25 21:04 kdkd