Harsha Vardhan Simhadri

@metastableB are you able to fix this using pandas?

@adityakusupati The first link looks broken

> It was because of the reorg - https://github.com/microsoft/EdgeML/blob/harsha/reorg/pytorch/edgeml_pytorch/trainer/fastTrainer.py @adityakusupati Can you test the fastmodel in PR 123, and see if you are happy. If everything you need is there,...

@metastableB I am a bit concerned about the complexity of step 1 in README.md. It was not straightfoward for me. Can we try another way of setting up MXChip? Like...

In evaluation, the dataset is not available. For T2, the index can store a copy of the data (or a compressed version) as part of the 1TB limit on index...

> should be ok now Thanks for addressing the requests. One more thing: could you please add an entry for random-xs dataset to https://github.com/harsha-simhadri/big-ann-benchmarks/blob/main/.github/workflows/benchmarks.yml

@NJU-yasuo I tried to run bigann-1B on F32s_v2 VM. After downloading the index, the load failed with docker exception 137. Could it be that the index is too big for...

> @harsha-simhadri I don't have F32_v2 so indexes were evaluated on my own machine and Azure L8sv2 VM and seems work well. I wonder if the index is too big...

I see the following results for deep-1B. I will try bigann-1B again with your new index. any other datasets? algorithm,parameters,dataset,count,qps,distcomps,build,indexsize,queriessize,mean_ssd_ios,mean_latency,recall/ap kst_ann_t1,"FaissIVFPQ(nprobe=8,quantizer_efSearch=32)",deep-1B,10,69460.0592205934,9659.5173,1000000.0,59865324.0,861.8668724407197,0,0,0.53951 kst_ann_t1,"FaissIVFPQ(nprobe=48,quantizer_efSearch=256)",deep-1B,10,9614.604704258865,57403.6261,1000000.0,59865324.0,6226.4987320261,0,0,0.75211 kst_ann_t1,"FaissIVFPQ(nprobe=32,quantizer_efSearch=64)",deep-1B,10,16391.414553927127,38474.8955,1000000.0,59865324.0,3652.2365902616502,0,0,0.71569 kst_ann_t1,"FaissIVFPQ(nprobe=48,quantizer_efSearch=64)",deep-1B,10,11275.156238575122,57526.6362,1000000.0,59865324.0,5309.489530192565,0,0,0.7498199999999999 kst_ann_t1,"FaissIVFPQ(nprobe=48,quantizer_efSearch=128)",deep-1B,10,10596.212267624653,57439.5464,1000000.0,59865324.0,5649.690897841931,0,0,0.7518900000000001 kst_ann_t1,"FaissIVFPQ(nprobe=64,quantizer_efSearch=64)",deep-1B,10,8387.557877753705,76566.8522,1000000.0,59865324.0,7137.396232910729,0,0,0.76973 kst_ann_t1,"FaissIVFPQ(nprobe=32,quantizer_efSearch=32)",deep-1B,10,17307.739762289606,38570.0483,1000000.0,59865324.0,3458.875903047466,0,0,0.71013 kst_ann_t1,"FaissIVFPQ(nprobe=48,quantizer_efSearch=96)",deep-1B,10,10906.352580931023,57465.3714,1000000.0,59865324.0,5489.032520796204,0,0,0.75142 kst_ann_t1,"FaissIVFPQ(nprobe=32,quantizer_efSearch=128)",deep-1B,10,15204.074715687373,38428.8152,1000000.0,59865324.0,3937.4526315785406,0,0,0.7172 kst_ann_t1,"FaissIVFPQ(nprobe=16,quantizer_efSearch=64)",deep-1B,10,36452.96786826431,19283.6208,1000000.0,59865324.0,1642.262002269459,0,0,0.63873

@NJU-yasuo Here is the complete set of results I see. (benchmark) harshasi@f32node2:~/big-ann-benchmarks$ python3 eval/show_operating_points.py --algorithm kst_ann_t1 --threshold 10000 res.csv recall/ap algorithm dataset kst_ann_t1 bigann-1B 0.712190 deep-1B 0.751890 msspacev-1B 0.764542 algorithm,parameters,dataset,count,qps,distcomps,build,indexsize,queriessize,mean_ssd_ios,mean_latency,recall/ap...