Skip to content

Conversation

@kdkd
Copy link

@kdkd kdkd commented Apr 2, 2025

This patch does two things to assist users that make use of the manual peer discovery system.

First, it adds a comment that documents the format of the config file that --discovery-config-path is expecting. As far as I can tell, nowhere in the source or documentation shows how this file is supposed to look.

Second, it adds a command-line parameter called --get-device-capabilities that outputs the current device's device_capabilities block. We needed this to be able to automate creating the manual configuration file, since those values aren't really easy to determine outside of exo. Example:

# exo --get-device-capabilities
{
  "model": "Mac Studio",
  "chip": "Apple M3 Ultra",
  "memory": 524288,
  "flops": {
    "fp32": 54.26,
    "fp16": 108.52,
    "int8": 217.04
  }
}

This is a little hairy because it needs to execute BEFORE anything else can output to the console, but the asyncio loop hasn't been started until after other things have been printed out and all the device capability code is async. So I added a small function to helpers.py to create a tiny event loop. There's probably a better way of refactoring this so all the real argument processing and startup happens in main() instead of at the top level, but that seemed too invasive for such a small feature. Maybe config_uvloop could execute earlier, but I'm not sure of the history for why that's pulled out separately.

Add --get-device-capabilities that outputs the current device's
device_capabilities object in JSON form, to allow external tooling
to automate the creation of the manual config file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant