REST

REST copied to clipboard

thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value'

1

When I run > RAYON_NUM_THREADS=6 CUDA_VISIBLE_DEVICES=0 python3 rest_test.py I encounter the following error: > thread '' panicked at 'called Option::unwrap() on a None value', src/lib.rs:282:53 The first time, reinstalling the...

homehappily

The original code only checks whether start_of_indices is None, but it does not check whether end_of_indices is None. However, it still calls .unwrap() on end_of_indices later without verifying its validity....

Lihui-Gu

Hi,thank you for your work! there is a question about the result of the running: After running the Inference on MT-Bench `cd llm_judge RAYON_NUM_THREADS=6 CUDA_VISIBLE_DEVICES=0 python3 gen_model_answer_rest.py --model-path lmsys/vicuna-7b-v1.5 --model-id...

YouthLiuYS

Issue description: I am using the draftretriever.search function, and I have encountered a few issues.I'm sorry, I'm not very familiar with Rust, so it's difficult for me to understand the...

qingyiaaaaa

Lama2 70B-chat is not supported?

2

When I run: ``` RAYON_NUM_THREADS=6 CUDA_VISIBLE_DEVICES=0,1,2,3 python3 gen_model_answer_rest.py --model-path /models/LLAMA-2-series/llama-2-70b-chat --model-id llama-2-70b-chat --datastore-path ../datastore/datastore_chat_small.idx ``` I get: (delete try and except) ``` Traceback (most recent call last): File "/rest/llm_judge/gen_model_answer_rest.py", line...

Chlience