Albert Tseng

Hi @oobabooga, is the blacksamorez aqlm model an official aqlm model (do they have a repo?) or someone's attempt at quantizing with their code? I've been trying to find an...

@oobabooga btw I just updated the quip-sharp repo with the latest code. The latest models are on HF and preprint is on arxiv as well.

Correct. It looks like AQLM also had some updated numbers for ICML vs what we had in our preprint but the latest QuIP# should still be better. I think the...

The updated QuIP# models are under the same model cards on HF so if you get bored you should be able to rerun eval on new QuIP# by just calling...

I'm also running into a similar issue, except with A6000s. With 1 A6000 and the rest of the weights on cpu, I get coherent text. With multiple A6000s, I get...

The canonical way to install QuIP# kernels is to install the fast-hadamard-transform package and build quiptools (in our codebase on github). We do not have a pypi package yet but...

I think it would be best to avoid duplicating code from the QuIP# codebase. The QuantizedLinear class is not standalone and relies on implementations in the codebook files (eg here...

Hi Younes, I’ll take a look at that, it definitely sounds interesting!

Hi Younes, will do. I got caught up with some other stuff but just released the updated quip# code and models today (https://github.com/Cornell-RelaxML/quip-sharp, https://arxiv.org/abs/2402.04396). Hoping to get integration going soon.

@younesbelkada we've finally started working on this, expect some progress in a week or so.