webarena
webarena copied to clipboard
Issue with the reddit website?
I am not super sure but I noticed something weird with the reddit website in the benchmark:
I was looking at task 29: "Tell me the count of comments that have received more downvotes than upvotes for the user who made the latest post on the DIY forum."
But when I go to the website, something doesn't look right. If I sort by hot, the website shows a submission that was made a day ago. But if I sort by new, the website shows the latest post was at least an year old. Am I not reading this correctly?
If the website is broken, it will confuse the agents and I am not sure we can have a valid evaluation.
cc: @afourney @cheng-tan