Hodge931

### Describe the issue Are texts corresponding to the field `patch` guaranteed to be correct answers in [princeton-nlp/SWE-bench_Lite_oracle](https://huggingface.co/datasets/princeton-nlp/SWE-bench_Lite_oracle)? In other words, to check if my evaluation environments are reliable, should...

Could I do multi-thread evaluation?

5

To speed up the evaluation, I would like to evaluate, say 64 examples in parallel with multiple threads. Does this affect the correctness of the evaluation? Thanks a lot!

Error installation

4

Running 'docker exec gitlab gitlab-ctl reconfigure` gives the following error ``` ffi-libarchive could not be loaded, libarchive is probably not installed on system, archive_file will not be available [2024-07-20T23:15:27+00:00] INFO:...

Fragile map site

1

The map site seems to be very fragile. I host it in AWS. However, the site becomes non-responsive after a few examples.