Andrew Balaschak

icon location United States icon location Washington State University, Pullman Bachelor of Science, Computer Science Minor in Mathematics

Results 2 comments of Andrew Balaschak

python data/openwebtext/prepare.py fails

> > If you indent the code from "tokenized = split_dataset.map(" to the bottom of the routine, and then wrap it with "if **name** == '**main**':", it should work. >...

Value Error when preparing openwebtext

I have a similar error: "ValueError: could not broadcast input array from shape (8665630,) into shape (4537790,)" on line 65 "arr[idx : idx + len(arr_batch)] = arr_batch"