How are the unlearning evaluation metrics computed?

[saptab]

Is the evaluation metric public?

Please share how the evaluation metric is computed

[fabianp]

We're working on making it open source. I expect it to take a couple of weeks

[saptab]

Hi Fabian, following up about the open source metric since its been two weeks :)

looking forward to learn about it and run it

[fabianp]

Unfortunately many people in the team have been busy with the ICML deadline, which I hadn't considered when I responded earlier. A more realistic timeline will be before the end of February. Apologies for the delay

On Sun, Feb 4, 2024 at 6:47 PM saptab @.***> wrote:

Hi Fabian, following up about the open source metric since its been two weeks :)

looking forward to learn about it and run it

— Reply to this email directly, view it on GitHub https://github.com/unlearning-challenge/starting-kit/issues/17#issuecomment-1925878766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACDZB5MZ5TV7RKQ65KRQTTYR7JSVAVCNFSM6AAAAABCDTTQT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVHA3TQNZWGY . You are receiving this because you commented.Message ID: @.***>

[yasser-h-khalil]

Hi @fabianp, when will the evaluation metrics be released?

[fugokidi]

Hi @fabianp Any news on evaluation metrics?

[fabianp]

it's finally online: https://github.com/google-deepmind/unlearning_evaluation