[Bug]: fix-me-experimental fails with "no such file or directory pyproject.toml"

[neubig]

Is there an existing issue for the same bug?

• [X] I have checked the existing issues.

Describe the bug and reproduction steps

The fix-me-experimental workflow is failing with a "no such file or directory" error on the OpenHands repo:

• Example Issue: https://github.com/All-Hands-AI/OpenHands/issues/6221
• Example Action: https://github.com/All-Hands-AI/OpenHands/actions/runs/12748739518

13:57:45 - openhands:INFO: resolve_issue.py:476 - Finished.
ERROR:root:  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line 652, in <module>
    main()
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line 631, in main
    asyncio.run(
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line [45](https://github.com/All-Hands-AI/OpenHands/actions/runs/12748739518/job/35529589182#step:10:46)9, in resolve_issue
    output = await process_issue(
             ^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line 195, in process_issue
    await runtime.connect()
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/runtime/impl/docker/docker_runtime.py", line 133, in connect
    self.runtime_container_image = build_runtime_image(
                                   ^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/runtime/utils/runtime_build.py", line 137, in build_runtime_image
    result = build_runtime_image_in_folder(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/runtime/utils/runtime_build.py", line 173, in build_runtime_image_in_folder
    lock_tag = f'oh_v{oh_version}_{get_hash_for_lock_files(base_image)}'
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.8/x64/lib/python3.12/site-packages/openhands/runtime/utils/runtime_build.py", line 312, in get_hash_for_lock_files
    with open(src, 'rb') as f:
         ^^^^^^^^^^^^^^^

ERROR:root:<class 'FileNotFoundError'>: [Errno 2] No such file or directory: '/opt/hostedtoolcache/Python/3.12.8/x[64](https://github.com/All-Hands-AI/OpenHands/actions/runs/12748739518/job/35529589182#step:10:65)/lib/python3.12/site-packages/pyproject.toml'

OpenHands Installation

Other

OpenHands Version

main

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

[enyst]

This looks related to or a consequence of https://github.com/All-Hands-AI/OpenHands/pull/5972

When trying to build the runtime image, it doesn't have the openhands project paths set as expected - it doesn't look for pyproject.toml in the project directory.

Note: the resolver builds with pip, while we use poetry. Now that it's integrated in the core openhands, we should probably consider migrating installations in the resolver to poetry, otherwise this discrepancy can introduce subtle, time-consuming bugs. (while I'm not sure it's the cause here, it's at least part of the difficulty to debug this kind of issue. The github action linked above installed "openhands experimental" with pip.)

[zchn]

@openhands-agent might be able to fix it: the bug is because when publishing to pypi, the workflow uses ./build.sh, and in ./build.sh, it copies both pyproject.toml and poetry.lock to the openhands directory at line 4 before running poetry build -v. However, when using the fix-me-experimental feature in the .github/workflows/openhands-resolver.yml workflow, it executes pip install git+https://github.com/all-hands-ai/openhands.git to install the openhands-ai package, so there is no cp pyproject.toml poetry.lock openhands executed before the package installation and consequently no pyproject.toml and poetry.lock in the package installed.

Instead of running cp pyproject.toml poetry.lock openhands in build.sh, the right way to add additional files into the build package should be changing the packages section in pyproject.toml from

{ include = "openhands/**/*" }

to

  { include = "openhands/**/*" },
  { include = "pyproject.toml", to = "openhands" },
  { include = "poetry.lock", to = "openhands" }

The cp pyproject.toml poetry.lock openhands command in build.sh should be removed since it is no longer needed.

See https://github.com/zchn/OpenHands/commit/5c78a59e8a8b81821bda5713fe14a901eb415ed0 for how I changed pyproject.toml based on the description above.

[enyst]

@openhands-agent Read @zchn 's comment, read the files it mentions, and implement the fix.

NOTE: this issue is https://github.com/All-Hands-AI/OpenHands/issues/6223 and you need to read all posts in it to understand the issue. You can use github API to do that.

[openhands-agent]

OpenHands started fixing the issue! You can monitor the progress here.

[openhands-agent]

A potential fix has been generated and a draft PR #6350 has been created. Please review the changes.

[zchn]

I created a fix for this in https://github.com/All-Hands-AI/OpenHands/pull/6658

[enyst]

This has been fixed by https://github.com/All-Hands-AI/OpenHands/pull/6658 thanks to @zchn .