flame icon indicating copy to clipboard operation
flame copied to clipboard

Adds mounts to fly backend

Open benbot opened this issue 1 year ago • 3 comments

I'm working on getting some elastic LLM inference set up using phoenix, fly, and FLAME and needed volumes attached to my flame workers so that I don't need to download an 8gb+ model on startup.

I added support for the mounts field in the machines api (https://fly.io/docs/machines/working-with-machines/)

Though it seems like fly has a bug where this isn't working right now. I keep getting 500s when passing in that argument. I made a thread on their forum to track it https://community.fly.io/t/machine-api-returning-500-when-passing-in-mounts/17336

benbot avatar Dec 20 '23 15:12 benbot

Quick update. Apparently the volumeId is a required field, so i'm going to work on a change to this to grab a volumeId from existing un-attached volumes with matching names, and use those ids to build the mounts array

benbot avatar Dec 21 '23 01:12 benbot

Alright, now the fly backend will attempt to find a volumeId that matches the name specified in the mounts config option.

If it finds enough it'll add those ids in as volume in the fly machine request.

This gets around the 500 that's thrown by fly if you don't include the volume id.

benbot avatar Dec 21 '23 17:12 benbot

These are great suggestions!

I'll get them in tomorrow

benbot avatar Dec 30 '23 03:12 benbot