Feature request: Do not fail when caching on non-Python repositories

[kdeldycke]

Description:

I am trying to use setup-python in a non Python repository. But I soon as I activate the cache, it ends up with this error:

Run actions/[email protected]
  with:
    python-version: 3.12
    cache: pip
Installed versions
  Successfully set up CPython (3.12.1)
Error: No file in /home/runner/work/awesome-iam/awesome-iam matched to
[**/requirements.txt or **/pyproject.toml], make sure you have checked out the target repository

Thats because the project I check out just before has no **/requirements.txt or **/pyproject.toml file.

And the reason it has no such file is that the actions/checkout and setup-python steps are located in a reused workflow.

Action version: [email protected]

Platform:

• [x] Ubuntu
• [x] macOS
• [x] Windows

Runner type:

• [x] Hosted
• [ ] Self-hosted

Tools version: Any version of Python.

Repro steps:

Here is the source code of my reused workflow: https://github.com/kdeldycke/workflows/blob/f50130aabec1fe713a548e3296f723732c79a178/.github/workflows/lint.yaml#L26-L35

And here is the result of this invokation: https://github.com/kdeldycke/awesome-iam/actions/runs/7592900227/job/20682781487#step:3:13

Expected behavior: I expect setup-python not to fail in the absence of a **/requirements.txt or **/pyproject.toml file.

Actual behavior: The step fails with the following message:

Run actions/[email protected]
  with:
    python-version: 3.12
    cache: pip
Installed versions
  Successfully set up CPython (3.12.1)
Error: No file in /home/runner/work/awesome-iam/awesome-iam matched to
[**/requirements.txt or **/pyproject.toml], make sure you have checked out the target repository

[kdeldycke]

Maybe this issue will be addressed by:

• https://github.com/actions/setup-python/issues/751
• https://github.com/actions/setup-python/pull/762

[kdeldycke]

To bypass current setup-python behavior, I created a step that is adding an empty requirements.txt if missing:

      - name: Hack setup-python cache
        # XXX Create an empty requirements.txt if this file (or pyproject.toml) doesn't exist.
        # This work around and issue with setup-python for non-Python projects, which ends up with ends up
        # with this error:
        #
        #   Run actions/[email protected]
        #     with:
        #       python-version: 3.12
        #       cache: pip
        #   Installed versions
        #     Successfully set up CPython (3.12.1)
        #   Error: No file in /home/runner/work/awesome-iam/awesome-iam matched to
        #   [**/requirements.txt or **/pyproject.toml], make sure you have checked out the target repository
        #
        # This has been reported at: https://github.com/actions/setup-python/issues/807
        # In the future this might be addressed by: https://github.com/actions/setup-python/pull/762
        # or https://github.com/actions/setup-python/issues/751
        if: hashFiles('**/requirements.txt', '**/pyproject.toml') == ''
        run: |
          touch ./requirements.txt

Source: https://github.com/kdeldycke/workflows/commit/662a971ef26023b87498fae7613a740d9148f61b

[HarithaVattikuti]

Hello @kdeldycke Thank you for creating this issue. We will investigate it and get back to you as soon as we have some feedback.

[aparnajyothi-y]

Hello @kdeldycke, Thank you once again for creating this issue. We have investigated the issue and actions/setup-python action is trying to find a requirements.txt or pyproject.toml file to cache the Python dependencies, but it's not able to find it. This error is encountered because setup-python is designed to cache Python dependencies and it expects these files to be present in Python projects. If the project is a non Python project and does not have any Python dependencies to cache, no need to activate the cache in setup-python. We can remove the cache: pip line from the workflow file. However, if we have Python dependencies that need to cache, we need to ensure that the project has a requirements.txt (for pip) or pyproject.toml (for pipenv) file. The actions/setup-python action will then be able to find these files and cache the dependencies. If you're using a reusable workflow, ensure that the repository being checked out in the actions/checkout step does indeed have these Python dependency files. Here is a sample workflow snippet for using setup-python: steps:

• uses: actions/checkout@v4

• uses: actions/[email protected] with: python-version: 3.12 If the files do exist in your repository but were not fetched during the actions/checkout step, you could try to adjust this step in your workflow. By default, the actions/checkout action only fetches a single commit. If your files are not in the latest commit, they won't be fetched. Fetch all history by setting fetch-depth: 0 as per this document

• name: Checkout code uses: actions/checkout@v2 with: fetch-depth: 0

  Please let us know if any further clarification required.

[kdeldycke]

Thanks @aparnajyothi-y for the reply. What you are describing is how setup-python is currently working. But I think I wasn't clear on my initial request.

What I'm looking for is to discuss the possibility for setup-python to support caching in reuseable workflow, and by extension, support caching of remote requirements.txt files.

[deslaughter]

@aparnajyothi-y This may be a silly question, but why does setup-python require that that checkout use the fetch-depth: 0 option? Checking out the entire history instead of just the last commit adds several minutes to the CI run. It's cheaper to disable the caching provided by setup-python than to use fetch-depth: 0 in my case.

[aparnajyothi-y]

Hello @kdeldycke, setup-python action doesn't directly have any support for caching. However, caching dependencies in GitHub workflows, including Python dependencies specified in a requirements.txt file, can typically be done using the actions/cache action.For reusable workflows, you can define a workflow in one repository and reference that workflow in other repositories. However, the actions/cache action currently has some limitations when used in reusable workflows.

As for caching of remote requirements.txt files, this would require some additional steps in your workflow:

• Fetch the remote requirements.txt file, perhaps using curl or wget.
• Generate a cache key from the contents of the requirements.txt file, such as by hashing the file's contents.
• Use this cache key with the actions/cache action. setup-action action itself does not provide a built-in caching mechanism. However, you can use the actions/cache action in conjunction with setup-python to cache dependencies, including Python packages installed with pip.

Here's an example of how you might use actions/cache to cache pip packages:

steps:

• uses: actions/checkout@v2
• uses: actions/setup-python@v2 with: python-version: '3.8'
• name: Cache pip packages uses: actions/cache@v2 with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip-
• name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt

Hello @deslaughter, The fetch-depth: 0 option in the actions/checkout action is used to fetch all history for all branches and tags. This is often necessary when we need the full git history to perform certain operations, like calculating a version number based on the number of commits, generating a changelog, or similar tasks. However, as we've noted, fetching the entire history can add time to your CI run. Setup-python doesn't inherently require fetch-depth : 0. The need to fetch all history would be specific to your workflow and the tasks you're performing. If the workflow doesn't require the full git history, you can use a shallow clone (like fetch-depth :1 to get only the latest commit) to save time and resources. This would not affect the functionality of setup-python, which is primarily concerned with setting up a specific version of Python in the runner environment.

[aparnajyothi-y]

Hello @kdeldycke, please confirm if the above-provided information clarifies your request. Thank you.

[kdeldycke]

@aparnajyothi-y no. Your answer is really low-quality and is just trying to avoid discussing the issue. Are you a LLM-based bot?

[aparnajyothi-y]

Hello @kdeldycke, We have explained the work arounds that we can use with action/setup-python and actions/checkout. If the actual request is to implement this feature from setup-python, it needs more investigation. We will get back to you on this once we got some feedback. Please feel free to share your thoughts on this.