clinicalml
Reported results
Hi,
I ran the simulation using the provided configuration and observed results that appear to differ from the results reported in your paper. Despite my efforts to investigate the cause of this discrepancy, I have been unable to determine why these results differ.
I have used the following configuration:
p_alpha=[0.3] p_lambda=[1e-4] n_in=[3] n_out=[3] nonlin='elu' lrate=[1e-3] batch_size=[100] dim_in=[200] dim_out=[100] batch_norm=[0] normalization=['divide'] imb_fun=['wass'] experiments=1000 reweight_sample=[1] split_output=[1] rep_weight_decay=[0] dropout_in=1.0 dropout_out=1.0 rbf_sigma=0.1 lrate_decay=0.97 decay=0.3 optimizer='Adam' wass_lambda=10.0 wass_iterations=10 wass_bpt=1 use_p_correction=0 iterations=3000 weight_init=[0.1] outdir='results/example_ihdp' datadir='data/' dataform='ihdp_npci_1-1000.train.npz' data_test='ihdp_npci_1-1000.test.npz' pred_output_delay=200 loss='l2' sparse=0 varsel=0 repetitions=1 val_part=0.3
and I got the following results:
Mode: Train | Pehe | Bias_ate | Rmse_fact | Rmse_cfact | Rmse_ite | Objective | Pehe_nn
0 | 0.785 +/- 0.052 | 0.219 +/- 0.031 | 1.135 +/- 0.045 | 1.188 +/- 0.020 | 1.164 +/- 0.021 | 0.994 +/- 0.036 | 4.895 +/- 0.613
Mode: Valid | Pehe | Bias_ate | Rmse_fact | Rmse_cfact | Rmse_ite | Objective | Pehe_nn
0 | 0.884 +/- 0.084 | 0.224 +/- 0.031 | 1.363 +/- 0.091 | 1.206 +/- 0.032 | 1.201 +/- 0.032 | 1.995 +/- 0.402 | 5.661 +/- 0.799
Mode: Test | Pehe | Bias_ate | Rmse_fact | Rmse_cfact | Rmse_ite | Pehe_nn
0 | 0.833 +/- 0.056 | 0.236 +/- 0.032 | 1.287 +/- 0.065 | 1.182 +/- 0.024 | 1.183 +/- 0.022 | 5.456 +/- 0.682
Thank you for your attention to this matter, and I look forward to your response.
