Ditto P S comments

Repositories
Issues
Comments

Results 55 comments of


                                            Ditto P S

[Misc] Support adapter scaling to all replicas

@Jeffwan Are you referring to adding a replica count in the ModelAdapter and use that for scheduling? eg: ``` spec: replicas: 3 # Only load on 3 pods podSelector: matchLabels:...

[Misc] Support adapter scaling to all replicas

@Jeffwan, I have rebased. Could you take a look?

Support content array in completions

Thanks for the update, the other PR seems resolved the issue. Closing this one.

ModelAdapter seems to be working abnormally

I have a similar issue. I'm trying to load 2 lora in the same deployment as per the yaml below. In that, qwen-code-lora is getting loaded, but the second one...

ModelAdapter seems to be working abnormally

I found the issue in the controller logs and resolved it by switching to a different Lora adapter. Currently the failure of LoRA loading is not clear. Specific error can...