minGPT icon indicating copy to clipboard operation
minGPT copied to clipboard

play_math AdditionDataset.__get_item__ return value?

Open SpeedCoder5 opened this issue 2 years ago • 0 comments

In the case of train_dataset[0], wherein self.permutation_array[0] contains 4717, why does get_item return x, y as (tensor([4, 7, 1, 7, 0, 6]), tensor([-100, -100, -100, 0, 6, 4])) and not (tensor([4, 7, 1, 7]), tensor([-100, -100, -100, -100, 0, 6, 4])) or (tensor([4, 7, 1, 7, -100, -100, -100]), tensor([-100, -100, -100, -100, 0, 6, 4]))

This question is not about the implementation of the function, rather it is about how the return value is used with minGPT. Is minGPT only trying to predict the the last digit, i.e. '4'? Not the last 3 digits '064'? Why are the last 3 digits not entirely excluded from x ? Why does y not include the first digit masked out?

SpeedCoder5 avatar Mar 25 '22 13:03 SpeedCoder5