kalomaze

Dynamic chunking option to reduce latency and overhead

4

### Issue Type Feature Request ### vc client version number N/A ### OS N/A ### GPU N/A ### Clear setting yes ### Sample model yes ### Input chunk num yes...

GGUF path handling with periods is incorrect

1

On my Windows machine, TinyLlama inferences properly, but the model is not properly stored as .gguf format due to the period in the model title.

To be merged for when https://github.com/ggerganov/llama.cpp/pull/6445 is merged

8-Bit DoRA training with FSDP doesn't work, but 4-bit QDoRA does / peft_use_dora is ignored?

3

### Please check that this issue hasn't been reported before. - [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports. ### Expected Behavior With 8-bit LoRA enabled and...

[Feature Request] - Koboldcpp probability

2

Being able to see probability on the kobold API backend would be a nice addition

'Hacker Mode' theme

1

- Generated text will be green, user text will be grey. - Terminal style font (monaco). - Rest of the theme is using a grayscale palette.  Currently there is...