flame
flame copied to clipboard
Adds mounts to fly backend
I'm working on getting some elastic LLM inference set up using phoenix, fly, and FLAME and needed volumes attached to my flame workers so that I don't need to download an 8gb+ model on startup.
I added support for the mounts
field in the machines api (https://fly.io/docs/machines/working-with-machines/)
Though it seems like fly has a bug where this isn't working right now. I keep getting 500s when passing in that argument. I made a thread on their forum to track it https://community.fly.io/t/machine-api-returning-500-when-passing-in-mounts/17336
Quick update. Apparently the volumeId is a required field, so i'm going to work on a change to this to grab a volumeId from existing un-attached volumes with matching names, and use those ids to build the mounts array
Alright, now the fly backend will attempt to find a volumeId that matches the name specified in the mounts config option.
If it finds enough it'll add those ids in as volume
in the fly machine request.
This gets around the 500 that's thrown by fly if you don't include the volume id.
These are great suggestions!
I'll get them in tomorrow