lorax

lorax copied to clipboard

LightLLM supports Multimodal llm, when will lorax?

1

### Model description https://github.com/ModelTC/lightllm/pull/266 Will there be vision llm support in Lorax soon? ### Open source status - [X] The model implementation is available - [X] The model weights are...

Trickster85

Is there any plan to support dynamic lora for qwen/chatglm models?

21

### Feature request Cool job! I have successfully run mulit-lora with llama2-70b. I would like to ask if the author has any plans to support other models, such as qwen,...

KrisWongz

Add support for additional custom models

1

### Feature request - [ ] Flash NeoX - [ ] Flash RW (Falcon) - [ ] Flash SantaCoder - [ ] Bloom - [ ] T5 - [ ]...

tgaddair

Can the basemodel be loaded directly from the local system without connecting to Hugging Face? It cannot be launched because there is no connection.

3

### Model description `model=Qwen/Qwen-7B-Chat volume=/hub_models/Qwen-7B-Chat/ docker run --gpus=1 --shm-size 1g -p 8080:80 -v $volume:/data \ ghcr.nju.edu.cn/predibase/lorax:latest --model-id $model` if load the local base model directly will be very well ###...

Loovelj

tgaddair

Multiple base models

4

### Model description Hi - I'm new to LoRAX. Just started reading the docs and github. Here's my situation. I have four locally hosted foundation (base) Open Source LLMs in...

sekhar-hari

Server error: This model was initialized with the adapter xxx and therefore does not support dynamic adapter loading. Please initialize a new model instance from the base model in order to use the dynamic adapter loading feature.

4

### System Info * Latest official docker image (ghcr.io/predibase/lorax:87412e1) * www.runpod.io platform (GPU containers) * GPU pod with 2xA100 80G. ### Information - [X] Docker - [ ] The CLI...

avoskresensky

Do you support miqu?

2

### Feature request Support the 'miqu' model. ### Motivation I am not sure if mique is actually the Mistral or Mixtral architecture or something else, but supposedly it may be...

runvnc

Windows Compatibility

1

### Feature request Tried to get this working on windows but ultimately ran into dead ends running the make files. I'm guessing windows just isn't supported yet. If anyone has...

TheLocalLab

# What does this PR do? ### TODO: - [ ] finish rust part to launch gui and pass in port - [ ] update docker to build the gui...

magdyksaleh