arrmansa

Hi, thanks for bringing this up > Am I doing something wrong, or severe reduction of quality is a consequence of RAM/VRAM memory savings? The vram savings don't cause a...

I've found some alternate pytorch checkpoints from https://rentry.org/jaxflawlessvictory > Easy Setup: > > Local KoboldAI-ready Monolithic Pytorch checkpoint file: > Checkpoint Converted by Author (.7z) > > https://mega.nz/file/z8QARTYI#rpjb54-rQh-76hHVEapfLfvNohj-R-_YZp21X4g5QHI > >...

> I'd like to point out that your timings (1.6 s /token) matches the timings I'm getting on a CPU only server with Hugginface Transformers library (which uses the 24GB...

Not sure what happened here, but it wasn't working for me today when I tried it. These are the steps to make it work. 1. Don't use the .exe, use...