david-jk

@MuhammadShifa It will be possible to run this on the CPU once support is added to llama.cpp and someone releases 4-bit (or lower) quantized weights. You will need around 256...