gujing

node-role.kubernetes.io/master label will be removed in v1.24, and w…

2

/kind feature

support update --data in kserve serving job

1

/kind feature support update --data in kserve serving job Fix #1048

1. submit kserve job ``` arena serve kserve \ --name=test \ --image=nginx \ --data="opt-pvc:/work/" \ "sleep 7d" ``` 2. update data ``` arena serve update kserve \ --name=test \ --image=nginx...

Can DeepSpeed-MII inference on multi gpus with only 1 replica?

2

I have 2 nodes, each with a 16GB GPU. And I want to run the llama-2-13b-hf model on these 2 nodes with 1 replica. cat /job/hostfile: ``` deepspeed-mii-inference-worker-0 slots=1 deepspeed-mii-inference-worker-1...

How to set trust_remote_code=True in pipeline

2

Hi, how to set trust_remote_code=True in mii pipeline? Code ``` from mii import pipeline pipe = pipeline("/root/Qwen-7B-Chat") pipe.tokenizer.tokenizer.eos_token_id = 151643 output = pipe(["DeepSpeed is"], max_new_tokens=128, do_sample=False) print(output) ``` Output ```...

**Describe the bug** InferenceSerivce must have either the storageUri or the storage.path **To Reproduce** Steps to reproduce the behavior: ``` apiVersion: "serving.kserve.io/v1beta1" kind: "InferenceService" metadata: name: "sklearn-iris-zibai" namespace: modelmesh-serving annotations:...