James comments

Results 4 comments of


                                            James

[Bug]: api/completion endpoint returns 200 but body contains error

Screenshots attached. ![completionerror_hoppscotch](https://github.com/infiniflow/ragflow/assets/65193532/b6c80d60-6fdd-4796-833d-5f840aebe2dd) ![completionerror](https://github.com/infiniflow/ragflow/assets/65193532/4c4ba087-b481-45d8-b11b-6bbb0c4d1472)

[Bug]: api/completion endpoint returns 200 but body contains error

I have tried both latest and dev. They all have the same issue. All the other endpoints, including the ones that were not working before are functioning normally, i.e. list_chunks....

[Bug]: api/completion endpoint returns 200 but body contains error

I have worked around this issue but the bug remains. The issue is with the 'stream' option which if set to True will cause the `retval 500`. For now I...

VLLM / OpenAI Compatible endpoint support

I am also facing issues with both VLLM and Llama-stack. VLLM seems to allocate too much KV memory and gets OOM errors from CUDA. Llama-stack seems to rely on `models/sku_list.py`...