Model uses label to generate predictions during training?

[tibnb545]

The input_ids tensor that is provided as input for the language modeling task contains the ground-truth label. Doesn't this mean the model uses the label (in addition to the other information in the input_ids tensor) to make predictions during training? I know the label is also given to the model in the lm_labels tensor and this is the tensor that the model checks its prediction against in order to calculate loss but if the model is making its prediction based on input_ids then it is making predictions with knowledge of what the correct label is. Is this correct? Or am I missing something. The only other way I can see that this might work is if the model somehow uses the -1 (mask) values in lm_labels to see which part of input_ids it should use to make the prediction (which presumably would make the model only consider elements in input_ids that correspond to elements in lm_labels that are equal to -1 since these elements are not part of the label).

Also, its not clear how the multiple-choice prediction task is done. I know each data instance contains multiple versions of the input_ids tensor and one of them contains the true label while the others contain distractor labels. Does the model look at each of these input_ids tensors and try to select the one that contains the true label out of all the options? If this is the case, since the input_ids tensor that contains the correct option is always the last one in the list of options, wouldn't the model just learn to always select the last option (similar to always predicting the same class for each instance) instead of actually learning anything useful?

I would really like to understand how the model works in these respects so I can create a custom dataset to train on so thank you for taking the time to read these questions.