Siamese-LSTM
Siamese-LSTM copied to clipboard
aditya1503
Prediction calibration step
Hi aditya1503, thanks for sharing your code. It was stated in your paper (p. 4) that a prediction calibration step was implemented to convert the predictions into values similar to the input. I copied below the exact text in your paper:
Due to the simple construction of our similarity function, the predictions of our model are constrained to follow the exp(−x) curve and are thus not suited for these evaluation metrics. After training our model, we apply an additional nonparametric regression step to obtain better-calibrated predictions (with respect to MSE). Over the training set, the given labels (under original [1, 5] scale) are regressed against the univariate MaLSTM g-predicted relatedness as the sole covariate, and the fitted regression function is evaluated on the MaLSTM-predicted relatedness of the test pairs to produce adjusted final predictions. We use the classical local-linear estimator discussed in Fan and Gijbels (1992) with bandwidth selected using leave-one-out cross-validation. This calibration step serves as a minor correction for our restrictively simple similarity function (which is necessary to retain interpretability of the sentence representations).
I have two clarifications:
- Could you please point out to me which part in your code this is? Sorry, I'm not sure in which part of the code it is implemented.
- Would converting the input labels to lie between [0,1] at the start have the same effect?((relatedness_score) - 1) / 4
Cheers, Kurt
@kurtespinosa I have the same questions, have you already solve those questions?
