Avishai Elmakies

### Describe the bug I used the finetuning tutorial to add orion to my code. i use hp context at the start of the code. I trained a model and...

### Feature request There are some models such that their forward pass doesn't get position_ids. e.g. we can see that OPTModel doesn't get position_ids, while GPTJModel does get position_ids. most...

Add position ids in forward pass to opt model

1

# What does this PR do? This pull request adds position_ids to the forward of ```OPT``` in a similar fashion to ```gemma``` and ```llama```. #32937 Some models didn't have an...

Add option to ignore keys/columns when loading a dataset from jsonl(or any other data format)

10

### Feature request Hi, I would like the option to ignore keys/columns when loading a dataset from files (e.g. jsonl). ### Motivation I am working on a dataset which is...

Dataset uses excessive memory when loading files

12

### Describe the bug Hi I am having an issue when loading a dataset. I have about 200 json files each about 1GB (total about 215GB). each row has a...