py-soft-impute icon indicating copy to clipboard operation
py-soft-impute copied to clipboard

Imputing train and test sets

Open YasminFathy opened this issue 4 years ago • 0 comments

Hi Travis, It seems fit() method in your implementation is equivalent to fit() and transform() methods in sklearn.impute. How could it be possible to fit into the training set and transform into the testing set?

clf = SoftImpute(J=2, lambda_=0.0)
 X_train_imp = clf.fit(X_train)
X_test_imp = clf.predict(X_test)

X_test_imp will be exactly X_train_imp if we don't make copyto=True. Should it be?

clf = SoftImpute(J=2, lambda_=0.0)
 X_train_imp = clf.fit(X_train)
fit.predict(X_test, copyto=True)
X_test

However, using the copyto=True means that we copy the value calculated in X_train into X_test and this is not the way the soft-impute algorithm works. We assume here that both of the training and testing sets are in order with the same sample size. Any thoughts on train/test set fitting and transforming?

Thanks, Yasmin

YasminFathy avatar Apr 13 '20 03:04 YasminFathy