Sagar Desai

icon indicating a website link https://sdcodehub.github.io/

icon company NVIDIA icon location Banglore icon location Active in LLM, Agents, RL space

Results 2 issues of Sagar Desai

datasets.utils.info_utils.ExpectedMoreSplits: {'validation'}

1 comment

``` ╰─$ python llama.py /datadrive/models/Llama-2-13b-chat-hf c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors /datadrive/models/Llama-2-13b-chat-hf-gptq/llama-2-13b-4bit-gs128.safetensors Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:02

Using pytrec_eval.RelevanceEvaluator, do we sort the results internally or use as it is provides?

1 comment

I am using ``` evaluator = pytrec_eval.RelevanceEvaluator(qrels, {map_string, ndcg_string, recall_string, precision_string}) scores = evaluator.evaluate(results) ``` Now result is `{qid: {docid: embeddding_score, docid: embeddding_score} }` so we sort the results using...