AlpinDale comments

Results 170 comments of


                                            AlpinDale

[Feature]: Implementation of DRY sampler.

Might be interesting to add. I currently don't have the bandwidth to tackle this. I may have the time after the 0.5.4 release, but if someone else wants to take...

[Bug]: Problem loading EXL2 in rc_054

Exl2 is currently broken in the rc_054 branch. Please read the PR description at #481

[Bug]: Issue when loading Mistral-Nemo-Instruct-2407-Q5_K_M

Fixed as of v0.6.0

[Bug]: LORA not working after commit e3f2ea4 "make punica kernels work with rocm" on rc_054 branch

Thanks for reporting. Does this only happen when you attempt launching a lora?

feat: TorchAO floating point quantization

I couldn't cleanly merge my changes with your latest commit, I'll add the SplitK myself in a bit (this is still a WIP)

[Usage]: start aphrodite in docker with tensor parallel

We don't directly take command-line arguments in the docker launch command. You will have to supply them as environment variables - please see the `.env` file in the `docker` directory...

[Usage]: start aphrodite in docker with tensor parallel

v0.6.0 has changed the docker to take arguments directly as CLI args. Please see the docker section in the documentation, or the snippet in the readme.

[Bug]: Using SillyTavern slow down aphrodite generation speed.

That's really weird, I haven't noticed this before. @sgsdxzy any ideas? I can't think of a reason why this would even happen.

[Bug]: Using SillyTavern slow down aphrodite generation speed.

Is this still an issue?

[Bug]: Llama 3.1 outputs gibberish when --kv-cache-dtype fp8 but AWQ model works fine

Are you on the main branch? The Llama 3.1 RoPE extension method has been added in the `rc_054` branch. Please try building that branch. As for the memory issues, I...