delphiRo
delphiRo
Could you please share modified https://github.com/NVIDIA/open-gpu-kernel-modules/blob/81fe4fb417c8ac3b9bdcc1d56827d116743892a5/src/common/shared/inc/g_vgpu_chip_flags.h Did you measure llama bench before nvidia-driver patch and after?
> > Could you please share modified https://github.com/NVIDIA/open-gpu-kernel-modules/blob/81fe4fb417c8ac3b9bdcc1d56827d116743892a5/src/common/shared/inc/g_vgpu_chip_flags.h > > Did you measure llama bench before nvidia-driver patch and after? > > In fact, I can't use this card on...
Am I understand right that this patch didn't affect the llama performance on Windows too? Did you build the llama source with fma disable for windows with official driver and...
> > Am I understand right that this patch didn't affect the llama performance on Windows too? Did you build the llama source with fma disable for windows with official...
> > > [@delphiRo](https://github.com/delphiRo) I can tell you my email address: [[email protected]](mailto:[email protected]). But I'm just an amateur and may not be of much help. > > > > > >...
> Nouveau It seems that Nouveau doesn't even load itself as a module for non VGA dev. Isn't it?
How to disable prefix prefill?
It seems that the export VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1 is not working solution in my case. I check on AMD Instinct Mi50 Rocm 6.3.4 and it log that all fp8 formats are not...
I Also check on AMD Instinct MI50 of Rocm 6.2.4. The same problem is also here when enabled the 32-40K context too
@simonw @jeethu @Sing-Li @philippgille