Xuanning Zhou

AttributeError: module 'lm_eval.tasks' has no attribute 'ALL_TASKS'

2

met ``` Traceback (most recent call last): File "/mnt/sharedata/hdd/zhouxn/wanda/main.py", line 112, in main() File "/mnt/sharedata/hdd/zhouxn/wanda/main.py", line 102, in main results = eval_zero_shot(args.model, model, tokenizer, task_list, num_shot, accelerate) File "/mnt/sharedata/hdd/zhouxn/wanda/lib/eval.py", line...

Hi, I noticed that you're using lm_eval v0.2.0 as an evaluation flow for more tasks, however, in v0.4.0 and later lm_eval added more datasets and made major changes to the...

TypeError: 'ListWrapper' object is not iterable

3

I met: ``` File "/mnt/sharedata/hdd/zhouxn/ICLCP/group_distance_k.py", line 33, in group_with_distance retriever = TopkRetriever(data, ice_num=3, batch_size=args.batch_size, index_split='train', test_split='test') File "/mnt/sharedata/hdd/zhouxn/ICLCP/openicl/icl_retriever/icl_topk_retriever.py", line 78, in __init__ self.index = self.create_index() File "/mnt/sharedata/hdd/zhouxn/ICLCP/openicl/icl_retriever/icl_topk_retriever.py", line 86, in...

plan for add power tuning to quantile point estimate

4

Hi @aangelopoulos Thank you for your clearly code. I was wondering if you have any plans to add power tuning to quantile estimation? Thanks