GenNAS icon indicating copy to clipboard operation
GenNAS copied to clipboard

Published 20 hours ago • verified publisher icon leeyeehoo

A question about nasbench1_search_20samples.json

Open zhengjian2322 opened this issue 2 years ago • 3 comments

Hello，I really enjoyed your paper! But I have a question about the file nasbench1_search_20samples.json, how do you sample the architectures in the file on the NASBench101.

zhengjian2322 avatar May 04 '22 15:05 zhengjian2322

@zhengjian2322 Hi, thank you for your notice! So we get all the archs from NASBench101 (If I didnt remember wrong NASWOT or ZeroProxyNAS provide the method in their github) and randomly sample 20 archs uniformly to generate nasbench1_search_20samples.json.

leeyeehoo avatar May 05 '22 16:05 leeyeehoo

Thanks. How does the number of the archs sampled effect the spearman‘s Rho in the proxy task search(4.3 in the paper)？Why do you choose 20？

zhengjian2322 avatar May 06 '22 17:05 zhengjian2322

Sorry, I don't use a larger number of architectures for proxy task searching in this part. I just use 20 as default. Maybe it's a good direction for further exploration :)

leeyeehoo avatar May 06 '22 17:05 leeyeehoo