Matt Psaltis

Swagger / OpenAPI and dotnet client for the api

10

Came across this amazing implementation today! Great to see C# and dotnet getting some awesome platforms like this nice work. Wanted to suggest swagger enabled documentation and possibly a dotnet...

Add MaxItemsToReturn to the request model to allow specifying values other than 100

3

https://github.com/OmniSharp/omnisharp-roslyn/blob/7299d0836af701b61dbbeb823c6a1efbb3010867/src/OmniSharp.LanguageServerProtocol/Handlers/OmniSharpWorkspaceSymbolsHandler.cs#L49 Pretty please?

Ray-LLM Head with VLLM Head throws configuration error

2

It looks like ray is receiving log_stats from VLLM in the remote call and then amending the config to explicitly add log_stats again? ``` (ServeController pid=501) RuntimeError: Traceback (most recent...

For example: https://huggingface.co/intfloat/e5-mistral-7b-instruct Currently it looks like these models do not support the token_type_ids property which is mandatory in the embedding warmup phase: https://github.com/ray-project/ray-llm/blob/master/rayllm/backend/llm/embedding/embedding_engine.py#L112C14-L112C28 Are you open to PRs on...

Hey team, hope you're all having a fantastic week and thanks for such an amazing contribution! In the 0.5.0 release, some files appear to be missing: For example, https://github.com/ray-project/ray-llm/blob/master/serve_configs/thenlper--gte-large.yaml References...

Multiple models second models always request GPU: 1

2

Using the instructions here: https://github.com/ray-project/ray-llm#how-do-i-deploy-multiple-models-at-once I'm trying to host two models on a single A100 80G. Two bundles are generated for the placement group: ``` {0: {'accelerator_type:A100': 0.1, 'CPU': 1.0}...

Constant instability, Service unavailable errors, System cannot find the file specified.

9

I'm trying to work out if this is specifically related to our machines, our setup using podman desktop or something else I haven't put my finger on. Essentially how stable...