carbon-assessment-with-ml
carbon-assessment-with-ml copied to clipboard
amazon-science
Query: Speed up evaluation with faiss search
Hi tried using the FAISS inner product similarity metric on the code in evaluation.ipynb The existing code took 25 minutes on the 6k annotated dataset. Whereas the FAISS implementation took just 19 seconds. The accuracies are quite different but comparable. I wanted to understand if this is a good contribution that can be made as a PR. The comparable accuracy, and 60X increased speed might be beneficial to test multiple sentence similarity models.
Top-1 accuracy w.r.t NAICS codes: 0.6466165413533834 Correct predictions: 3698, Total Products: 5719 Top-1 accuracy w.r.t BEA codes: 0.7518796992481203 Correct predictions: 4300, Total Products: 5719
Top-1 accuracy w.r.t NAICS codes (FAISS): 0.6284315439762196 Correct predictions: 3594, Total Products: 5719 Top-1 accuracy w.r.t BEA codes (FAISS): 0.7361426822871131 Correct predictions: 4210, Total Products: 5719
