zero-cost-nas icon indicating copy to clipboard operation
zero-cost-nas copied to clipboard

Published 20 hours ago • verified publisher icon SamsungLabs

Reproducing results for NAS-Bench-ASR

Open rohanmittal73 opened this issue 1 year ago • 0 comments

Currently I am trying to reproduce results on the NAS-ASR benchmark however I have been unsuccessful so far. It would be really helpful if you could answer some questions I had regarding this:

  1. I am querying architectures from the ASR space and directly running the given proxies on the architectures (similar to how it's done in the code for NASBench201 (nasbench201_pred.py)). Are there any intermediary steps involved specific to the ASR case that I am missing?

  2. Is the original ASR architecture (https://github.com/SamsungLabs/nb-asr) modified in any way or are any of the parameters changed before running the given implementation of the proxies?

  3. Are the zero-cost proxy implementations modified in any way for the ASR case? Specific to Synflow, are all the layers (Conv1d, Linear, LSTM and Layernorm) considered for pruning while calculating the metric for a particular architecture?

In case I am completely off the mark, I would be really grateful if you could provide a broad outline for how I can go about reproducing these results.

rohanmittal73 avatar Apr 04 '23 16:04 rohanmittal73