openreview-expertise
openreview-expertise copied to clipboard
openreview
Integrates ACL Sentence Piece Model for calculating affinity scores
Only files in the expertise/models/acl_scorer need to be reviewed. Everything else is formatting changes.
My changes are on top of harold's changes. I can take my changes out in a separate branch if we want to release this first.
Hello, I just wanted to check if this was stable and going to be merged into the main branch?
@neubig this branch is stable, however, we are parallely working on another project to expose expertise as an API. We plan to release both of them together, most probably next week. While we finish working on that, it'd be great if you can use this branch to test whether the model's working correctly or not.
Hi! We've been looking at this, but seeing some unexpected behavior (namely the overall scores are lower than we expect). I'd like to dig into this a bit more, but this commit has so many files changed it's a bit hard for me to tell which parts I should be looking at. I wonder if you could rebase this branch so only the changes that actually are relevant to the ACL scorer would be visible?
Also, to clarify what the issue is here: when running the ACL reviewer matching code we have previously gotten relatively high matching scores of 0.7 or so, but when running the code here we get lowish scores of 0.3. We're not sure exactly where the difference is coming from, but it'd be nice to be able to read the changed parts and try to debug.
Hi @neubig , we are working on this, it seems @purujitgoyal fixed formatting issues in the same PR. We are creating a new branch with only the changes needed for ACL model.
This is the draft PR with changes needed to run the ACL scores: https://github.com/openreview/openreview-expertise/pull/80
Feel free to make code suggestions.