[Usage]: Any tips on troubleshooting Quant-LLM

[GHBigD]

Your current environment

The output of `python env.py`

How would you like to use Aphrodite?

I run the latest Docker on full weights model and it runs perfect. Add the --quantization fp6 switch and it goes brain dead .. spewing nonsense and then ultimately looping a three or four word phrase over and over.

[GHBigD]

Upon tinkering, I figured out it is coherent using KoboldAI Classic in SillyTavern but not Text Completion Default or Aphrodite? I went side by side between the two presets to make sure samplers matched as best I could, but there are so many more things getting sent under text completion instead of KoboldAI Classic.