liveck

The experimental results cannot be replicated. script do not work!

2

when calling run_job.bash ``` majority/majority_exp.py", line 294, in run_experiment _, queries, _, keys, Q, K, _ = model(classes) ValueError: not enough values to unpack (expected 7, got 3) ``` it...

ValueError: can't optimize a non-leaf Tensor (param.is_leaf=False,param.retains_grad=False)

1

My model works fine with adamw_bnb_8bit. When i switched to galore_adamw_8bit with 'all-linear', an exception is raised 'can't optimize a non-leaf' ``` Seq2SeqTrainingArguments( output_dir = model_name_or_path, save_strategy = 'no', logging_steps...

It seems that bipe_alibi is not working yet.

1

It seems that bipe_alibi is not working yet. `get_ape_embeddings` returns a tuple, which is different from embed_tokens. All codes since here do not work. ```Python if self.config.rpe_type == "bipe_alibi": inputs_embeds...