Bhimraj Yadav

> looking good @bhimrazy so far! We might have to run some benchmarks to verify that we don't lose performance because of multiprocessing synchronization. But really good approach 😄 Thanks,...

Sure, that sounds great!

Hi @aniketmaurya, I am looking for abit of guidance for cancelling the task which already started in `run_single_loop` (non streaming case). > specifically this part from `run_single_loop` ```python x =...

Closing this PR. Due to the complexity involved, the streaming and non-streaming cases will be handled separately in new PRs (in more better and cleaner way). You can find the...

Hi @TheLukaDragar, I was able to reproduce the issue you reported—thanks for bringing it up! > btw you can ignore those files ignored warnings; ### **Workaround:** - You can refer...

Hi @MatejRojec, thanks for reporting the issue! In the latest versions, you also need to pass `TokensLoader` to optimize token handling before streaming: ```python # This informs LitData that we're...

Closing this issue since it has been resolved. Please feel free to reopen if you run into any related problems or have further questions. cc: @Borda

> @bhimrazy mind have a look here, pls 🐿️ Sure @Borda — thanks for the ping! I’ll take a look shortly. 🙌

Hi @deependujha, could you pls check if you're able to add the changelog? ```markdown - Fixed `AdvancedProfiler` to handle nested profiling actions for Python 3.12+ ([#20809](https://github.com/Lightning-AI/pytorch-lightning/pull/20809)) ``` I started getting...

Hi @arkohut, You can add an additional endpoint by implementing a LitSpec API, similar to the [OpenAISpec](https://github.com/Lightning-AI/LitServe/blob/6bdd8f3ed29256e003140aeff109304f8f52dfeb/src/litserve/specs/openai.py#L260). Currently, it only takes a single spec.