Questions abut the search process

[JoinWei-PKU]

Hi, @ildoonet

Thanks for your great work, and it has inpired me a lot. Recently, I'm trying to reproduce the search results. When I utilize the ray.tune.hyperoptsearch method as the search method, I cannot get higher accuracy after augmenting the validation data compared with "without augmentation". However, as mentioned in your paper, the results should be better.

Is the phenomenon is normal? If not, how can you utilize the ray package to implement the search process?

Looking forward to your reply.

Thanks.

[Shaw95]

Hi, @ildoonet

I've run into the same problem when trying to reproduce the search process. The augmentated valid accuracy during hyperoptsearch was consistently lower than the best valid accuracy when training with DM. Perhaps the problems lie in my implementation of ray hyperoptsearch , so it would be much appreciated if you could explain more details about how you implemented ray to complete your search or consider release your search code. I believe it should be very helpful to the academic circles.

Thanks for your enlightening work. Wait for your reply,

[ildoonet]

@JoinWei-PKU @Shaw95 See our released codes.

search.py