lolcats issues

Clarification for the paper's needle-in-a-haystack section

In the needle-in-a-haystack section of your paper, you mentioned: "However, linearizing with passkey samples (LoLCATs Llama 3 8B (Passkey)) recovers 100% accuracy." Does this step involving lora-finetuning with passkey samples?...

yiyousong

Reported Scores in Paper may be low

Table 6 shows Lolcats distilled from Llama 3.1 8B gets 69.7% acc on Winogrande (w/ Llama 3.1 8B gets 73.5% acc) Then Table 3/4 show that Lolcats distilled from Llama...

lukemcdermotttt

ThunderKittens Hedgehog doesn't support A100 (cannot use lolcats_llama_window_tk_gen)

2

Hi there, thanks for this open-source codebase which is detailed and almost works fine for training and evaluation. But as mentioned in your [Blog Part2](https://hazyresearch.stanford.edu/blog/2024-10-14-lolcats-p2), we're also fighting for A100s,...

z76316

Using TK kernels results in a bad model

1

Hi, Thanks for open sourcing your code and model weights. As the title says, I am trying to use TK kernels with pre-linearized llama 3.1 8b model and unable to...

ajtejankar

I'm having trouble running the distill_llama.py.

1

I think there's one experiment missing in the experiment config folder: eval_Scrolls.yaml, which calls OurTrainer in the finetune_seq2seq.py in the eval phase, which uses dataset scrolls, but doesn't precede the...

foreverboy1989