runner-images icon indicating copy to clipboard operation
runner-images copied to clipboard

Linux user changing between versions

Open williamho123 opened this issue 7 months ago • 7 comments

Description

When using actions/cache@v4 to restore a cache on disk, the tar command is erroring out with messages like:

/usr/bin/tar: ../../../../../opt/hostedtoolcache/Python/3.10.17/x64/bin: Cannot utime: Operation not permitted
/usr/bin/tar: ../../../../../opt/hostedtoolcache/Python/3.10.17/x64/bin: Cannot change mode to rwxr-xr-x: Operation not permitted

Upon further inspection, it appears that the Action is executing as the runner user but the owner of the directory in question is packer. This is leading to the permissions issues.

Is this new behavior expected? Previous versions of the runner restored cache successfully.

Also, this appears to affect Ubuntu 24.04 runner image version 20250427.1.0 but we don't see the issue on the newer 20250504.1.0. In the same repository, we were using the newer image version up until today, when for some reason, the Github hosted runners started using the older version.

Platforms affected

  • [ ] Azure DevOps
  • [x] GitHub Actions - Standard Runners
  • [ ] GitHub Actions - Larger Runners

Runner images affected

  • [ ] Ubuntu 22.04
  • [x] Ubuntu 24.04
  • [ ] macOS 13
  • [ ] macOS 13 Arm64
  • [ ] macOS 14
  • [ ] macOS 14 Arm64
  • [ ] macOS 15
  • [ ] macOS 15 Arm64
  • [ ] Windows Server 2019
  • [ ] Windows Server 2022
  • [ ] Windows Server 2025

Image version and build link

20250427.1.0

Is it regression?

Yes - 20250504.1.0

Expected behavior

The cache restore to work successfully, and the owner of the directory to be the same user as the one executing the Action (runner).

Actual behavior

The cache restore fails with permissions errors from tar.

Repro steps

  1. Setup a workflow that sets up Python 3.10 with actions/setup-python@v5.
  2. Also use the cache action to cache the Python install: actions/cache@v4
  3. Run the workflow for the first time. It will succeed because it is an empty cache.
  4. Run the workflow again. The cache restore will fail if runner image is version 20250427.1.0 (I'm not sure how to reliably select the version here).

Example:

- name: Setup Python
  id: setup-python
  uses: actions/setup-python@v5
  with:
    python-version: 3.10

- name: Cache
  uses: actions/cache@v4
  id: cache
  with:
    path: ${{ env.pythonLocation }}
    key: <replace-with-my-cache-key>

williamho123 avatar May 13 '25 18:05 williamho123

Forgot to post screenshots of the different users and versions

Successful version

Image Image

Failing version

Image Image

williamho123 avatar May 13 '25 18:05 williamho123

Hi @williamho123, Thank you for bringing this issue to our attention. We will look into this issue and will update you after investigating.

hemanthmanga avatar May 14 '25 04:05 hemanthmanga

Hi @williamho123 - Could you please try once to rerun your workflows and let us know.

Prabhatkumar59 avatar May 16 '25 06:05 Prabhatkumar59

Hi @Prabhatkumar59 -- I re-ran our workflow, and I still see that my current user is runner but the packer user is the owner of the directory.

When actions/cache@v4 tries to restore the cache, I still get the same permissions issues. However, the entire workflow does not fail anymore.

  1. How come this is the case? What changed in the backend?
  2. Is the user owning the directory expected to be packer?

williamho123 avatar May 16 '25 21:05 williamho123

Hi @williamho123 - Thanks for the input!, we will look into this further and will update you accordingly, thank you!

Prabhatkumar59 avatar May 19 '25 08:05 Prabhatkumar59

Hay @williamho123, we fixed the directory owner. The next version of the image will have this fix.

Alexey-Ayupov avatar May 22 '25 13:05 Alexey-Ayupov

Hay @williamho123, we fixed the directory owner. The next version of the image will have this fix.

Hi @Alexey-Ayupov -- thanks for the update. Could you provide some more details about what images this was affecting?

From my original issue description, 20250427.1.0 was problematic but a newer version of 20250504.1.0 was working. Does that mean it was already fixed in 20250504.1.0?

Or does this have to do with the differing versions of the Runner Image Provisioner? (I am not sure what that is)

williamho123 avatar May 22 '25 18:05 williamho123

As per above comment from @Alexey-Ayupov - We have fixed the directory owner. Please try to update your sync process and use latest image version. We are closing this issue for now, thanks.

Prabhatkumar59 avatar Jun 04 '25 13:06 Prabhatkumar59

@Prabhatkumar59 @Alexey-Ayupov This is not fixed. We should re-open the issue.

For a recent run from this morning, I have this snippet as part of my CI workflow.

current_user=$(whoami)
python_install_owner=$(ls -ld /opt/hostedtoolcache/Python/3.10.17/x64 | cut -d' ' -f3)
  
if [[ "$python_install_owner" != "$current_user" ]]; then
  echo "WARNING: Python install owner is not '$current_user'. It is '$python_install_owner'."
fi

I got the warning:

Image

Runner versions Image

How come it is still showing a different user?

Please try to update your sync process and use latest image version

How do I do that? I am using Github Hosted Runners and do not have control over the image version.

williamho123 avatar Jun 05 '25 19:06 williamho123