Badis

@daboe01 I have the RTN quantized on llama cpp and the GPTQ quantized on the webui but it would be hard to compare the 2 as they are a bit...

I just tried it and it works like a charm!! GPTQ quantized models will be the standard and thanks to you the CPU users can enjoy it aswell Thanks again...

@Belluxx Yeah, the file went bigger, maybe it could be more optimized Idk, only @comex has the explaination about that 😅

Yeah it's a bit slower when using the GPTQ: Regular RTN quantization: ``` main: seed = 1 system_info: n_threads = 14 / 16 | AVX = 1 | AVX2 =...

@MillionthOdin16 Sorry I messed up, I didn't use the same models for both at the begining, I fixed that and arrived into a different conclusion (I modified the post in...

@MillionthOdin16 What I meant by "deterministic" runs is that I fixed the seed to 1, and that alone is supposed to give you the exact same output regardless of anything...

@PriNova Nah, when you fix the seed (I put seed = 1) you'll get the same output everytime. Try it by yourself, put a fixed seed + high temperature and...

@nusu-github wouldn't this work? ``` cmake_minimum_required(VERSION 3.8) project(YourProjectName LANGUAGES CXX) # Set C++ standard set(CMAKE_CXX_STANDARD 17) # Add the source files set(SOURCE_FILES main.cpp) add_executable(YourProjectName ${SOURCE_FILES}) # Enable SSE3, F16C, and...

@nusu-github I'm using this cmd command to compile in MSVC: ``` Remove-Item -Path ".\Windows-build\*" -Force -Recurse cmake -S . -B Windows-build/ -D CMAKE_BUILD_TYPE=Release cmake --build Windows-build/ --config Release ``` What...

@nusu-github thanks I appreciate! But I looked at the CMakeList.txt and I don't think there's clang activated in it, so it will alaways create exe binary through MSVC ``` D:\Large...