Question about Ranking nats

[pprp]

after run code:

cd ranking_nats
python get_model_score_nats.py

I got:

kendall tau begin
BossNAS: KendalltauResult(correlation=-0.534180602248828, pvalue=0.0)
(-0.7180607093955225, 0.0)
SpearmanrResult(correlation=-0.7341493538551311, pvalue=0.0)

[changlin31]

This seems a normal result, except the minus. Referring this to @Trent-tangtao , who performed the ranking experiments.

[tangtaogo]

after run code:

cd ranking_nats
python get_model_score_nats.py

I got:

kendall tau begin
BossNAS: KendalltauResult(correlation=-0.534180602248828, pvalue=0.0)
(-0.7180607093955225, 0.0)
SpearmanrResult(correlation=-0.7341493538551311, pvalue=0.0)

I think you means the architecture rating accuracies on cifar100 dataset are shown in Tab. 4. There is no problem that the result of the code is cifar10, you can refer to the result Table 6: Architecture rating accuracy on NATS-Bench SS with CIFAR datasets. The results of cifar100 will be release later if you need.