sotabench-api

sotabench-api copied to clipboard

Bumps [requests](https://github.com/psf/requests) from 2.24.0 to 2.31.0. Release notes Sourced from requests's releases. v2.31.0 2.31.0 (2023-05-22) Security Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of Proxy-Authorization...

dependabot[bot]

Hello, I try to use sotabench-api package, and it gives me the below error as sotabench.com is not currently working. Could you help me regarding this? Traceback (most recent call...

MEssam711

I tried uploading imagenet using the `sb dataset upload` command to work around the changes that required us to upload the dataset ourselves. I cannot figure out how to get...

rwightman

Allow to run without cache

1

As discussed in Slack, having option to ignore cache can be useful sometimes. In particular, if I run model someone already run, the result is returned from cache. However, the...

Randl

Run hash should give a warning for empty/all-zero inputs

1

In some cases the first batch might be too small, or simply contain no interesting predictions (eg empty or all zeros). Run hash calculation should catch this, produce a warning...

rstojnic

RJT1990

RJT1990

Use PEP-8 code formatting

1

Use PEP-8 standard for formatting code, perhaps run the whole code through a formatter or install the pre-commit hooks ?

rstojnic

Add latest version of torchvision to requirements and eliminate copy-pasted code

1

We should add latest pytorch/torchvision installation from github into requirements, then eliminate all copy-pasted code.

rstojnic

TODO sotabench lib: - remove benchmark() function from `benchmark.py` - move deps to requirements - evaluation.json should be made if some ENV variable is set, otherwise pprint something - for...

rstojnic