Fix sympy version for specific instance

[SmartManoj]

Fixes #265

Modify the build_instance_image function in swebench/harness/docker_build.py to check for the instance ID sympy__sympy-21612 and install antlr4-python3-runtime==4.7.2 if it matches.

• Add a check for the specific instance ID sympy__sympy-21612
• Append the installation command for antlr4-python3-runtime==4.7.2 to the repo_script_list if the instance ID matches

---

For more details, open the Copilot Workspace session.

[john-b-yang]

This fix is not necessary. The gold patch prediction works just fine.

$ ./test.sh
/opt/miniconda3/envs/sweb/lib/python3.10/runpy.py:126: RuntimeWarning: 'swebench.harness.run_evaluation' found in sys.modules after import of package 'swebench.harness', but prior to execution of 'swebench.harness.run_evaluation'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Using gold predictions - ignoring predictions_path
Running 1 unevaluated instances...
Base image sweb.base.x86_64:latest already exists, skipping build.
Base images built successfully.
No environment images need to be built.
Found 1 existing instance images. Will reuse them.
Running 1 instances...
1 ran successfully, 0 failed: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:19<00:00, 19.55s/it]
All instances run.
Cleaning cached images...
Removed 0 images.
Total instances: 1
Instances submitted: 1
Instances completed: 1
Instances incomplete: 0
Instances resolved: 1
Instances unresolved: 0
Instances with empty patches: 0
Instances with errors: 0
Unstopped containers: 0
Unremoved images: 1
Report written to gold.gold.json

$ cat test.sh
python -m swebench.harness.run_evaluation \
    --predictions_path gold \
    --max_workers 1 \
    --run_id gold \
    --split test \
    --dataset_name princeton-nlp/SWE-bench \
    --instance_ids sympy__sympy-21612 \

In the future, if you have an issue, please provide a log like this one to prove that there is an issue with the instance. I appreciate your contributions, but please make sure to do an appropriate amount of auditing such that I can focus more on resolving the problem.

[SmartManoj]

Trajectory link

The package was not installed in the environment by default, and the latest version is not compatible.

parsed traj

---

Solved traj

[john-b-yang]

OpenHands maintains separate execution environments than SWE-bench.

Again, please please please run the SWE-bench code. If you are seeing an issue, please verify that this problem is actually reproducible with SWE-bench before posting.

[SmartManoj]

    --instance_ids sympy__sympy-21612 \

Is the next line --cache_level instance?

Reproduced in SWE bench:

C:\Users\smart>docker run -it sweb.eval.x86_64.sympy__sympy-21612:latest /bin/bash
(testbed) root@ce34aca68a6b:/testbed# python
Python 3.9.21 (main, Dec 11 2024, 16:24:11)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from sympy.parsing.latex import parse_latex
>>> parse_latex("\\frac{\\frac{a^3+b}{c}}{\\frac{1}{c^2}}")
/testbed/sympy/parsing/latex/_parse_latex_antlr.py:23: UserWarning: antlr4.error.ErrorListener module is not installed
  ErrorListener = import_module('antlr4.error.ErrorListener',
/testbed/sympy/parsing/latex/_parse_latex_antlr.py:62: UserWarning: antlr4 module is not installed
  antlr4 = import_module('antlr4', warn_not_installed=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/testbed/sympy/parsing/latex/__init__.py", line 35, in parse_latex
    return _latex.parse_latex(s)
  File "/testbed/sympy/parsing/latex/_parse_latex_antlr.py", line 65, in parse_latex
    raise ImportError("LaTeX parsing requires the antlr4 python package,"
ImportError: LaTeX parsing requires the antlr4 python package, provided by pip (antlr4-python2-runtime or antlr4-python3-runtime) or conda (antlr-python-runtime)
>>>

As the underlying issue is different, the library is not needed for evaluation but for reproducing the issue only.

---

OpenHands maintains separate execution environments than SWE-bench.

But the readme says different. Is it outdated?