Change evaluation metric

[wszhtc]

I set the parameters 'metric' to 'auc'. When training the model, each epoch will show train log_loss, test log_loss, test auc. Is it possible to show train auc?

[etveritas]

I think you miss the output information, if you change the metric to auc, you can see the Test AUC in console.

[wszhtc]

I think you miss the output information, if you change the metric to auc, you can see the Test AUC in console.

Yes, I can see 'Test AUC'. But my question is: is it possible to show 'Train AUC'?

[etveritas]

@wszhtc Sorry. I get it. The parameter of metric is for validation set or test set, and the loss is for the optimization procedure, for classification, it's cross entropy and for regression, it's squared loss. For now, doesn't support.

[ymkigeg]

I think you miss the output information, if you change the metric to auc, you can see the Test AUC in console.

Yes, I can see 'Test AUC'. But my question is: is it possible to show 'Train AUC'?

I meet the same problem, will this feature be added in future version?

[wszhtc]

@wszhtc Sorry. I get it. The parameter of metric is for validation set or test set

When I set validation set with 'model.setValidate' and train model with 'model.fit', I can see 'Test AUC' in training process. But when I test the model with 'model.setTest' and 'model.predict', it just show test loss and save the predicting result to file. Is it possible to see the Test AUC in predicting process?

Here is my code:

model = xl.create_fm() model.setTrain(train_local_path) param = {'task':'binary', 'lr':1, 'init': 0.5, 'k': 40, 'lambda':0.000001, 'metric':'auc', 'fold':5, 'epoch':20, 'stop_window':3, 'nthread':30} model.setTXTModel(model_txt_path) model.fit(param, model_bin_path) model.setTest(test_local_path) model.predict(model_bin_path, model_predict_path)