achervyakov
achervyakov
may be useful for researchers to get tokenier info. The results may depend on pad_token that may be absent (and replaced by UNK or EOS) which should be visible to...
The PR add support for new type of tasks - context-based tasks. Motivation. Some tasks and CoT strategies may require knowing the answer of the model for the previous question...
Hew feature - "smart truncation". This PR: - allow for different truncation modes, not only cutting off ALL tokens from the left (which is now the only available option, that...
Features: - disable `fewshot_as_multiturn` when `apply_chat_template` is not passed or `num_fewshots=0`. Why failing the run? For zero-shot setup multiturn==simple chat template, so no error at all. If chat_template is not...