Vinay Varma

icon indicating a website link https://vinayvarma.work/about/

icon location I want to build intelligent systems. Previously, I automated financial trading using Deep Reinforcement Learning at Niveshi.

Results 1 comments of Vinay Varma

Generation with Prefix-cache are slower than the ones without it ?

With the latest version(0.4.0), it seems we cannot enable prefix caching with Mistral-type models(sliding window attention). ``` if enable_caching and sliding_window is not None: raise NotImplementedError( "Sliding window is not...