Nikhil Ranjan issues

Repositories
Issues
Comments

Results 3 issues of


                                            Nikhil Ranjan

Plain text tagging

Hi, I am trying to use this model for inference but the plain text tagging comes out to be very weird. Input: "Hello, today is Monday but not Tuesday, maybe...

Typeerror while converting to streaming format

I am trying to convert Redpajama-github dataset to streaming format but getting the error as below. To replicate: python llm-foundry/scripts/data_prep/convert_dataset_json.py \ --path github/split1 \ --out_root github/split1 --split train \ --concat_tokens...

bug

How to get latest data ?

I think the PMC urls has not been updated since 2016. Is it possible to get bulk data latest one in jsonl format ?