building image sympy__sympy-20590: Environment image sweb.env.py.x86_64.c795f4b88616b8462021ed:latest not found for sympy__sympy-20590

[linzm1007]

Describe the issue

/usr/lib/python3.10/runpy.py:126: RuntimeWarning: 'swebench.harness.run_evaluation' found in sys.modules after import of package 'swebench.harness', but prior to execution of 'swebench.harness.run_evaluation'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) Using gold predictions - ignoring predictions_path Running 1 unevaluated instances... Base image sweb.base.py.x86_64:latest already exists, skipping build. Base images built successfully. Total environment images to build: 1 0%| | 0/1 [00:00<?, ?it/s]<class 'swebench.harness.docker_build.BuildImageError'>: Error building image sweb.env.py.x86_64.c795f4b88616b8462021ed:latest: The command '/bin/sh -c /bin/bash -c "source ~/.bashrc && /root/setup_env.sh"' returned a non-zero code: 1 Check (logs/build_images/env/sweb.env.py.x86_64.c795f4b88616b8462021ed__latest/build_image.log) for more information. Traceback (most recent call last): File "/data/linziming/code/SWE-bench-3.0.8/swebench/harness/docker_build.py", line 155, in build_image raise docker.errors.BuildError( docker.errors.BuildError: The command '/bin/sh -c /bin/bash -c "source ~/.bashrc && /root/setup_env.sh"' returned a non-zero code: 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/data/linziming/code/SWE-bench-3.0.8/swebench/harness/utils.py", line 92, in run_threadpool future.result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result return self.__get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/data/linziming/code/SWE-bench-3.0.8/swebench/harness/docker_build.py", line 161, in build_image raise BuildImageError(image_name, str(e), logger) from e swebench.harness.docker_build.BuildImageError: Error building image sweb.env.py.x86_64.c795f4b88616b8462021ed:latest: The command '/bin/sh -c /bin/bash -c "source ~/.bashrc && /root/setup_env.sh"' returned a non-zero code: 1 Check (logs/build_images/env/sweb.env.py.x86_64.c795f4b88616b8462021ed__latest/build_image.log) for more information. 0%| | 0/1 [01:21<?, ?it/s] 1 environment images failed to build. Running 1 instances... 0%| | 0/1 [00:00<?, ?it/s]Error building image sympy__sympy-20590: Environment image sweb.env.py.x86_64.c795f4b88616b8462021ed:latest not found for sympy__sympy-20590 Check (logs/run_evaluation/validate-gold/gold/sympy__sympy-20590/run_instance.log) for more information. 1 ran successfully, 0 failed: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 301.16it/s] All instances run. Cleaning cached images... Removed 0 images. Total instances: 1 Instances submitted: 1 Instances completed: 0 Instances incomplete: 0 Instances resolved: 0 Instances unresolved: 0 Instances with empty patches: 0 Instances with errors: 1 Unstopped containers: 0 Unremoved images: 0

Suggest an improvement to documentation

python3 -m swebench.harness.run_evaluation
--dataset_name /data/linziming/SWE-bench_Verified
--predictions_path gold
--max_workers 1
--instance_ids sympy__sympy-20590
--run_id validate-gold
--namespace ''

SWE-bench-3.0.8

[AMindToThink]

We spent a long time trying to understand an error and failed until we stumbled upon this PR. Please accept this pull request to the documentation!

[john-b-yang]

Hi @linzm1007 @AMindToThink thanks for raising an issue - I'm not quite understanding, what part of the documentation is misleading?

The officially recommended way to run SWE-bench Verified using this repository is

python3 -m swebench.harness.run_evaluation
--dataset_name princeton-nlp/SWE-bench_Verified
--predictions_path gold
--max_workers 1
--instance_ids sympy__sympy-20590
--run_id validate-gold

I would not recommend doing --namespace '' unless you want to build locally - is that your intention?

[AMindToThink]

Yes, it was unclear what namespace meant, and I did not realize I needed to use it. The resulting log errors made no suggestion that this was the solution.

[john-b-yang]

Ah ok - it's been added to the README, here