sileod comments

Repositories
Issues
Comments

Results 13 comments of


                                            sileod

Pretraining the deberta-v3 by larger context length.

Hi, I did a multi-task fine-tune with 1280 context length (1680 for small version) https://huggingface.co/tasksource/deberta-base-long-nli

Why expect Z in Adapter?

Hi, great question It is used here: https://github.com/sileod/tasknet/blob/c9f43604c2c8bc7e33e3301f7c05b5dc8f77c0eb/src/tasknet/utils.py#L210 But actually, it would be cleaner to have it in `adapt_model_to_task` I'll try to do it for the next release The general...

Using tasksource for question answering datasets

Hi, are you talking about Squad-like extractive question answering ? https://huggingface.co/datasets/rajpurkar/squad Or classification or multiple choice QA ? Tasksource only handles the latter currently. To train on other datasets, you...