FastChat
FastChat copied to clipboard
API or data blob access to leaderboard output
I asked on the Discord, and I believe that this isn't available today. If I am wrong, happy if you can direct me to the source.
At my company, I am responsible for compiling a list of (non-proprietary) models that our product is compatible with. I thought it would be nice if I could query the contents of the Leaderboard via an API, rather than picking through the list on a browser and copying and pasting.
Is the data that populates the current version of the leaderboard rankings available via an API, or an export of that data available as a flat-file in some manner?
There are examples of similar properties doing this, for example, MLperf releases their data (example )under Tableau, I'm hoping this isn't an unreasonable ask. I'm not asking for anything as fancy as Tableau, not even an API. A read-only CSV on AWS S3 or similar would be fantastic.
Right now, the site uses CloudFlare Captcha to prevent screen-scraping, which I respect. But it feels like making a limited set of data available would prevent people from either trying to circumvent captcha and/or copying and pasting a bunch of data by hand.
I think what I am asking is that the leaderboard-table-file (gradio_web_server_multi.py#L337) (docs) is copied to an S3 bucket, or even just served raw via the site's http server (sans captcha, please), so that we can look at the raw data.