Windows: python interpreter version equal to defined lambda runtime (pythonX.Y) should be available in system path

[baolsen]

Description

When on Windows and building a Python package in a Virtual Env, the package.py script produces an error such as below:

python interpreter version equal to defined lambda runtime (python3.10) should be available in system path

The error message is descriptive and the reason and workaround is as described in #487.

Basically, within the virtual environment in Windows there is python.exe but not python3.10.exe. For some reason on Linux there is a python3.10 which therefore prevents this issue, but not on Windows.

• [x] ✋ I have searched the open/closed issues and my issue IS listed - #487.

Versions

• Module version [Required]: ~> 7.4 specifically 7.4.0

• Terraform version: 1.6.3

• Provider version(s):

Terraform v1.6.3
on windows_amd64
+ provider registry.terraform.io/hashicorp/aws v5.51.1
+ provider registry.terraform.io/hashicorp/external v2.3.3
+ provider registry.terraform.io/hashicorp/local v2.5.1
+ provider registry.terraform.io/hashicorp/null v3.2.2
+ provider registry.terraform.io/hashicorp/random v3.6.2
+ provider registry.terraform.io/hashicorp/tls v4.0.5

Reproduction Code [Required]

Simplified code below.

module "lambda_to_start_execution" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 7.4"

  function_name = "${local.general_name}-lambda-to-start-execution"
  handler       = "main.lambda_handler"
  runtime       = var.lambda_runtime # is equal to "python3.10"

  # NOTE, if you are running on Windows and you get this:
  # python interpreter version equal to defined lambda runtime (python3.10) should be available in system path
  # Workaround is to copy .venv/Scripts/python.exe to .venv/Scripts/python3.10.exe

  timeout = 30

  create_package = true
  source_path = [{
    path             = "${path.module}/lambda-to-start-execution"
    pip_requirements = true
  }]

   ... more code ...
}

Steps to reproduce the behavior:

• Not using workspaces.

1. Install Python on Windows.

In the Python installation dir observe there is "python.exe" and "python3.10.exe"

2. Create and activate Python venv

python3.10 -m venv .venv or python -m venv .venv etc This is done as I have multiple Python versions installed and want to be sure the right one is available for testing and development.

When using VSCode the venv is automatically activated.

3. Observe contents of .venv/Scripts:

There is python.exe but not python3.10.exe.

PS C:\Users\me\VSCode\myproject> ls .\.venv\Scripts
                                                                                                                                                                                                                                                                                                                                                Directory: C:\Users\me\VSCode\myproject\.venv\Scripts                                                                                                            


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        2024/05/27     10:03                __pycache__
-a----        2024/05/27     10:02           2077 activate
-a----        2024/05/27     10:02           1006 activate.bat
-a----        2024/05/27     10:02          20655 Activate.ps1
-a----        2024/05/27     10:04         106364 black.exe
-a----        2024/05/27     10:04         106365 blackd.exe
-a----        2024/05/27     10:02            393 deactivate.bat
-a----        2024/05/30     09:17         106355 identify-cli.exe
-a----        2024/05/27     10:03           1734 jp.py
-a----        2024/05/27     10:03         106364 markdown-it.exe
-a----        2024/05/30     09:18         106353 moto_proxy.exe
-a----        2024/05/30     09:18         106354 moto_server.exe
-a----        2024/05/30     09:17         106350 nodeenv.exe
-a----        2024/05/27     10:03         106377 normalizer.exe
-a----        2024/05/27     10:02         106365 pip.exe
-a----        2024/05/27     10:02         106365 pip3.10.exe
-a----        2024/05/27     10:02         106365 pip3.exe
-a----        2024/05/30     09:18         106358 pre-commit.exe
-a----        2024/05/30     09:17         106365 py.test.exe
-a----        2024/05/27     10:03         106359 pygmentize.exe
-a----        2024/05/30     09:17         106365 pytest.exe
-a----        2024/05/27     10:02         263120 python.exe
-a----        2024/05/27     10:02         251856 pythonw.exe
-a----        2024/05/30     09:17         106382 virtualenv.exe

4. Attempt to terraform apply the module and you get the error

Expected behavior

The package.py script should work as there is a correct version of Python in the path. It is not called python3.10 instead it is called simply python

Actual behavior

│ Error: External Program Execution Failed
│
│   with module.infrastructure.module.lambda_to_start_execution.data.external.archive_prepare[0],
│   on .terraform\modules\infrastructure.lambda_to_start_execution\package.tf line 10, in data "external" "archive_prepare":
│   10:   program = [local.python, "${path.module}/package.py", "prepare"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ Program: C:\Users\me\VSCode\myproject\.venv\Scripts\python.exe
│ Error Message: Traceback (most recent call last):
│   File "C:\Users\me\VSCode\myproject\terraform\.terraform\modules\infrastructure.lambda_to_start_execution\package.py", line 1777, in <module>
│     main()
│   File "C:\Users\me\VSCode\myproject\terraform\.terraform\modules\infrastructure.lambda_to_start_execution\package.py", line 1773, in main
│     exit(args.command(args))
│   File "C:\Users\me\VSCode\myproject\terraform\.terraform\modules\infrastructure.lambda_to_start_execution\package.py", line 1540, in prepare_command
│     build_plan = bpm.plan(source_path, query)
│   File "C:\Users\me\VSCode\myproject\terraform\.terraform\modules\infrastructure.lambda_to_start_execution\package.py", line 815, in plan
│     pip_requirements_step(
│   File "C:\Users\me\VSCode\myproject\terraform\.terraform\modules\infrastructure.lambda_to_start_execution\package.py", line 687, in pip_requirements_step
│     raise RuntimeError(
│ RuntimeError: Python interpreter version equal to defined lambda runtime (python3.10) should be available in system PATH
│
│ State: exit status 1

Terminal Output Screenshot(s)

Unable to upload as the GitHUb upload URLs are blocked by corporate firewall.

Additional context

I think requiring the exact "python3.10" on the path as a preference is fine, as this works fine on Linux and is more explicit. But there should be a fallback added to check "python" exists and it has a required version, then proceed with the build (maybe add a warning).

Alternatively - probably better solution - a parameter added to the Terraform module to make it possible for a user to specify the path to the Python command themselves (and avoid the package.py maybe assuming the wrong thing)