The problem with Meta-DNN

[han-shi]

Awesome work! I trained net_predictor with default settings (Adding validation_split=.2). The val_correlation can achieve 0.819, which is much bigger than the number (0.784) in the paper. Is there anything wrong? Thanks.

[linnanwang]

Sorry just double check our experiments. Yes, we used NASBench for designing validation of meta-DNN. I think the discrepancy in evaluating the val_correlation is at "how you split the dataset". Did you randomize NASBench before splitting it? Please noted validation dataset can vary, resulting in different correlations. I believe the differences in validation dataset is causing the problem.

We'd also want to bring your attention to our another recent work, which significantly improves the sampling efficiency of AlphaX. https://linnanwang.github.io/latent-actions.pdf

[auroua]

@linnanwang Does the code of LaNAS have released?