LGTM.com - 3.10 is not a valid value for LGTM_PYTHON_SETUP_VERSION

[igo95862]

trafficstars

I am trying to make LGTM.com to use Python 3.10 instead of 3.8 because in 3.9 ArgumentParser.__init__ gained a new keyword exit_on_error but LGTM is not recognizing that.

This is my lgtm.yml

---

extraction:
  python:
    python_setup:
      version: '3.10'

This is the error I get on LGTM:

[2022-06-23 15:57:38] [build-stderr] SyntaxError: Illegal value for LGTM_PYTHON_SETUP_VERSION
[2022-06-23 15:57:38] [build-stdout] index:
[2022-06-23 15:57:39] [build-stderr] Traceback (most recent call last):
[2022-06-23 15:57:39] [build-stderr]   File "/opt/dist/python/tools/index.py", line 25, in <module>
[2022-06-23 15:57:39] [build-stderr]     buildtools.index.main()
[2022-06-23 15:57:39] [build-stderr]   File "/home/runner/work/semmle-code/semmle-code/target/thirdparty/python/build/extractor-python/buildtools/index.py", line 354, in main
[2022-06-23 15:57:39] [build-stderr]     
[2022-06-23 15:57:39] [build-stderr]   File "/home/runner/work/semmle-code/semmle-code/target/thirdparty/python/build/extractor-python/buildtools/discover.py", line 58, in get_version
[2022-06-23 15:57:39] [build-stderr]   File "/home/runner/work/semmle-code/semmle-code/target/thirdparty/python/build/extractor-python/buildtools/version.py", line 144, in best_version
[2022-06-23 15:57:39] [build-stderr] SyntaxError: Illegal value for LGTM_PYTHON_SETUP_VERSION
[2022-06-23 15:57:39] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/opt/work/lgtm-workspace/lgtm/extract.sh])

[aibaars]

The python_setup.version field only allows 2 or 3 . It's purpose is to indicate to the CodeQL analyzer whether the code should be analyzed as Python 2 or Python 3.

I think you can achieve what you want you could try installing python 3.9 from https://downloads.python.org/pypy/ , unpack it and put the bin directory on the PATH before the setup_python step is run. I think the following should work:

extraction:
  python:
    after_prepare:
      - wget https://downloads.python.org/pypy/pypy3.9-v7.3.9-linux64.tar.bz2 -O"${LGTM_WORKSPACE}/python3.9.tar.bz2"
      - tar -C "${LGTM_WORKSPACE}" -xjf "${LGTM_WORKSPACE}/python3.9.tar.bz2" 
      - PATH="${LGTM_WORKSPACE}/pypy3.9-v7.3.9-linux64/bin:$PATH"

Note: you can use the `Test analysis configuration" button to try out configuration changes, see:

• https://lgtm.com/projects/g/igo95862/bubblejail/logs/languages/lang:python
• https://lgtm.com/logs/4958582048bf479ddd6c47d20bcf96beb12009b0/lang:python

[mehdialiakbar31]

I am trying to make LGTM.com to use Python 3.10 instead of 3.8 because in 3.9 ArgumentParser.__init__ gained a new keyword exit_on_error but LGTM is not recognizing that.

This is my lgtm.yml

---

extraction:
  python:
    python_setup:
      version: '3.10'

This is the error I get on LGTM:

[2022-06-23 15:57:38] [build-stderr] SyntaxError: Illegal value for LGTM_PYTHON_SETUP_VERSION
[2022-06-23 15:57:38] [build-stdout] index:
[2022-06-23 15:57:39] [build-stderr] Traceback (most recent call last):
[2022-06-23 15:57:39] [build-stderr]   File "/opt/dist/python/tools/index.py", line 25, in <module>
[2022-06-23 15:57:39] [build-stderr]     buildtools.index.main()
[2022-06-23 15:57:39] [build-stderr]   File "/home/runner/work/semmle-code/semmle-code/target/thirdparty/python/build/extractor-python/buildtools/index.py", line 354, in main
[2022-06-23 15:57:39] [build-stderr]     
[2022-06-23 15:57:39] [build-stderr]   File "/home/runner/work/semmle-code/semmle-code/target/thirdparty/python/build/extractor-python/buildtools/discover.py", line 58, in get_version
[2022-06-23 15:57:39] [build-stderr]   File "/home/runner/work/semmle-code/semmle-code/target/thirdparty/python/build/extractor-python/buildtools/version.py", line 144, in best_version
[2022-06-23 15:57:39] [build-stderr] SyntaxError: Illegal value for LGTM_PYTHON_SETUP_VERSION
[2022-06-23 15:57:39] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/opt/work/lgtm-workspace/lgtm/extract.sh])

[igo95862]

The python_setup.version field only allows 2 or 3 . It's purpose is to indicate to the CodeQL analyzer whether the code should be analyzed as Python 2 or Python 3.

I think you can achieve what you want you could try installing python 3.9 from https://downloads.python.org/pypy/ , unpack it and put the bin directory on the PATH before the setup_python step is run. I think the following should work:

extraction:
  python:
    after_prepare:
      - wget https://downloads.python.org/pypy/pypy3.9-v7.3.9-linux64.tar.bz2 -O"${LGTM_WORKSPACE}/python3.9.tar.bz2"
      - tar -xjf "${LGTM_WORKSPACE}/python3.9.tar.bz2"
      - PATH="${LGTM_WORKSPACE}/pypy3.9-v7.3.9-linux64/bin:$PATH"

Note: you can use the `Test analysis configuration" button to try out configuration changes, see:

* https://lgtm.com/projects/g/igo95862/bubblejail/logs/languages/lang:python

* https://lgtm.com/logs/71e081a586508b25a6f149ef299863474b59f4de/lang:python

Thanks. I will look in to this.

Do you know what distro LGTM is running on? Is it Ubuntu 20.04? (I am going to try a PPA for ubuntu with more recent python package)

[aibaars]

Thanks. I will look in to this.

Note I forgot a -C $LGTM_WORKSPACE flag in the tar command above.

Do you know what distro LGTM is running on? Is it Ubuntu 20.04? (I am going to try a PPA for ubuntu with more recent python package)

I think it is 20.04, but I'm not sure LGTM permits adding any PPA. Still worth a try though.

[nuztalgia]

Just ran into this while trying to get LGTM to realize that kw_only is, in fact, "a supported parameter name of function dataclass".

Here's what I put in my lgtm.yml to make it use Python 3.10:

extraction:
  python:
    after_prepare:
      - "cd $LGTM_WORKSPACE"
      - "wget https://www.python.org/ftp/python/3.10.5/Python-3.10.5.tgz"
      - "tar -zxf Python-3.10.5.tgz"
      - "cd Python-3.10.5 && ./configure && make && mv python python3"
      - "PATH=\"${LGTM_WORKSPACE}/Python-3.10.5:$PATH\""
    python_setup:
      version: "3"

Results: https://lgtm.com/projects/g/nuztalgia/botstrap/rev/5999e5ef467e250312c3f1c50f76aa743e3c036a

Hopefully that's helpful to anyone having similar issues, but it's quite a janky workaround. Are there any plans to update the default version of python3 that LGTM uses? Currently it's 3.8, which is almost 3 years out of date. Alternatively, allowing users to specify which minor version of Python they'd like to use, as @igo95862 originally suggested, would be an awesome feature!

[hmakholm]

Closing since LGTM.com has been discontinued.