pyDVL
pyDVL copied to clipboard
Implement NTK scorer
Introduced in Zhaoxuan Wu, Yao Shu, and Bryan Kian Hsiang Low, “DAVINZ: Data Valuation Using Deep Neural Networks at Initialization,” in Proceedings of the 39th International Conference on Machine Learning (International Conference on Machine Learning, PMLR, 2022), 24150–76.
It's a scorer using the Neural Tangent Kernel on untrained networks with a correction to estimate generalisation error. Implementation available in the supp. materials of the paper.
(Maybe relevant, JAX library for NTK: https://github.com/google/neural-tangents, and paper about fast finite-width NTK computation)