Jeff Emanuel

I agree, it was quite annoying to get it to work in Python. Here is my solution: First, download the latest model files from https://github.com/GantMan/nsfw_model/releases/download/1.2.0/mobilenet_v2_140_224.1.zip and extract them somewhere. Set...

I don't know much C++, but I asked GPT4 to look at the Python GPTQ code and then modify the existing lama.cpp quantization code, and this is what it came...

Another example, showing how it suggests to implement it (let me know if these are wrong and I'll delete them! Just trying to be helpful and don't want to pollute...

For a version that only uses the stdlib: ``` #include #include #include #include #include // Perform Cholesky decomposition on a square matrix std::vector cholesky_decomposition(const std::vector &matrix, size_t size) { std::vector...

This has been working well for me, using a response from ChatGPT that I shortened by a couple sentences to save more of the context space (oddly, while this works...

If you quantized the 7B model to a mixture of 3-bit and 4-bit quantization using https://github.com/qwopqwop200/GPTQ-for-LLaMa then you could stay within that memory envelope.

You’re welcome to submit a PR for that feature. On Tue, Nov 21, 2023 at 9:08 PM shaojun ***@***.***> wrote: > for these doc format convertion, text summarization tasks, I...

Not really, but you’re welcome to submit a PR. On Fri, Nov 24, 2023 at 2:42 PM AYOUB EL MHAMDI ***@***.***> wrote: > Is there any plan to restructure the...

Weird, I haven't seen anything like that. But I've only run it under Linux.

Hey sorry for the delay in responding, I've been super busy. Will go through this more thoroughly when I have a chance. It's a bit of a pain to prepare...