[Bug]: Too many detected issues in github resolver

[neubig]

Is there an existing issue for the same bug?

• [X] I have checked the existing issues.

Describe the bug and reproduction steps

There are too many issues detected in the github resolver. For instance, running on this PR: https://github.com/All-Hands-AI/OpenHands/pull/5449

Finds many issues that are not mentioned anywhere in the PR:

$ poetry run python openhands/resolver/resolve_issue.py --repo all-hands-ai/openhands --issue-type pr --issue-number 5432
16:15:54 - openhands:INFO: issue_definitions.py:608 - Limiting resolving to issues [5432].
16:15:55 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6729: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6729
16:15:55 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6218: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6218
16:15:55 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6668: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6668
16:15:56 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6605: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6605
16:15:56 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6608: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6608
16:15:56 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6615: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6615
16:15:56 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6680: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6680
16:15:56 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6489: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6489
16:15:56 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6617: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6617
16:15:57 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6683: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6683
16:15:57 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6619: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6619
16:15:57 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6633: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6633
16:15:57 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6634: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6634
16:15:58 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6637: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6637
16:15:58 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6640: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6640
16:15:58 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6644: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6644
16:15:59 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6714: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6714
16:15:59 - openhands:WARNING: issue_definitions.py:597 - Failed to fetch issue 6590: 404 Client Error: Not Found for url: https://api.github.com/repos/all-hands-ai/openhands/issues/6590

We should write a test to reproduce this issue, and fix it.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

[openhands-agent]

OpenHands started fixing the issue! You can monitor the progress here.

[openhands-agent]

A potential fix has been generated and a draft PR #5457 has been created. Please review the changes.

[neubig]

I'm not sure why I saw this behavior before, but 5432 was not actually a pull request (it was a typo). Now with recent changes to the resolver this gives the expected behavior of returning this error:

ERROR:root:<class 'ValueError'>: No issues found for issue number 5432. Please verify that:
1. The issue/PR #5432 exists in the repository All-Hands-AI/OpenHands
2. You have the correct permissions to access it
3. The repository name is spelled correctly