ci(ubuntu): ci use ubuntu24.04

[lhdjply]

Description of the feature or fix

ci(ubuntu): ci use ubuntu24.04

Notes

• Update the Documentation if needed.
• Add Examples if relevant.
• Add Tests if applicable.
• If you added new options to lv_conf_template.h run lv_conf_internal_gen.py and update Kconfig.
• Run scripts/code-format.py (astyle version v3.4.12 needs to be installed) and follow the Code Conventions.
• Mark the Pull request as Draft while you are working on the first version, and mark is as Ready when it's ready for review.
• When changes were requested, re-request review to notify the maintainers.
• Help us to review this Pull Request! Anyone can approve or request changes.

[lhdjply]

@kisvegabor thorvg is error in ubuntu24.04

@kdschlosser @PGNetHun Micropython CI is error in ubuntu24.04

[kdschlosser]

Ubuntu 24.04 was released 30 days ago. It is not something we have done any work with. The issue is not so much with the version of Ubuntu as it is with the versions of the packages. The newer the version of Ubuntu the newer the versions for the packages.

The error in the MicroPython binding requires the binding to be updated to MicroPython 1.21.0 at a minimum in order to work. There is nothing I can do about that one. That's in @PGNetHun 's arena.

As far as the ThorVG error I am not sure. If ThorVG is an external lib that is being statically linked I would check what version is being used for Ununtu 22.04 and then update the CI to use that version for Ubuntu 24.04 if the version is available. If not then you are going to need to make the changes needed to get ThorVG working with LVGL. IDL if that PR would even be accepted considering Ubuntu 24.04 was released only 30 days ago.

You might get lucky and the version of ThorVG that is used in 22.04 might be available for 24.04. If it is then it is simply a matter of changing the CI to use the version that is used in Ubuntu 22.04

[kdschlosser]

There would be a lot of work that needs to be done to get the MicroPython version updated to 1.21.0, so that's probably not gonna happen in a day or 2. It will be a while.

[lhdjply]

update thorvg to 0.13.5 to fix the thorvg CI #6274

[lhdjply]

fix Micropython CI https://github.com/lvgl/lv_micropython/pull/81

[kisvegabor]

The CI failing with this strange error message:

Does it say anything to you?

[lhdjply]

The CI failing with this strange error message:

Does it say anything to you?

see https://github.com/actions/runner-images/issues/9977

[kisvegabor]

Probably we needed to duplicate the test-native part for 22.04 (32bit) and 24.04 (64bit). Let's use 22.04 here for now. We will rework how build tests and unit test are working later anyway. cc @XuNeo

[lhdjply]

In the local test, Ubuntu24.04 runs 32-bit CIs. I think we can wait, because in actions/runner-images,Ubuntu 24.04 is still a beta version, not an release version.There are more bugs. Let's see if this problem will be fixed. Like the last version,Ci run for a period of time after the direct error. After the actions/runner-images update, it was fixed.

[lhdjply]

In addition, the CI also reports an error during compiling idf.

[lhdjply]

Currently, it is not recommended to merge into the master branch. I propose temporarily merging into a new branch and periodically synchronizing it with the master branch's code. When the actions/runner-images/Ubuntu 24.04 stable version is released, we can then consider merging this new branch into master.

[kisvegabor]

Instead of creating a new branch, let's just leave this PR open.

[liamHowatt]

Should we bump the Python version to match the one that comes with Ubuntu 24.04? Currently CI uses 3.10, but Ubuntu 24.04 ships with 3.12. Running ./test/main.py under 3.12 emits a new warning. See #6509

[lhdjply]

Should we bump the Python version to match the one that comes with Ubuntu 24.04? Currently CI uses 3.10, but Ubuntu 24.04 ships with 3.12. Running ./test/main.py under 3.12 emits a new warning. See #6509

Okay, it has been updated to version 3.12. But the ESP-IDF compilation has encountered an errort.

[liamHowatt]

Thanks. After this gets merged in lv_micropython, we can try this

diff --git a/.github/workflows/build_micropython.yml b/.github/workflows/build_micropython.yml
index 1d54351d9..c80208d2f 100644
--- a/.github/workflows/build_micropython.yml
+++ b/.github/workflows/build_micropython.yml
@@ -47,7 +47,7 @@ jobs:
     - name: Setup ESP-IDF
       if: matrix.port == 'esp32'
       run: |
-        source tools/ci.sh && ci_esp32_idf44_setup
+        source tools/ci.sh && ci_esp32_idf522_setup
     - name: Build ESP32 port
       if: matrix.port == 'esp32'
       run: |

I confirmed locally that it gets past that issue.

[liamHowatt]

@lhdjply it was merged. You can update build_micropython.yml and the CI may pass :) thanks

[lhdjply]

@lhdjply it was merged. You can update build_micropython.yml and the CI may pass :) thanks

@liamHowatt Passed ESP-IDF, but encountering build failure for ESP32 port.

[liamHowatt]

I could not work around the issue, despite trying. Sorry, I think I was too quick to suggest using IDF 5.2.2. I'm trying to remember why the build broke when you added

    - name: Setup Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.12'

to build_micropython.yml. The Ubuntu 24.04 runner is supposed to come with 3.12 anyways.

I'll leave it up to you to revert the ci_esp32_idf44_setup and the Python version requirement, if you don't have any better suggestions.

[lhdjply]

I could not work around the issue, despite trying. Sorry, I think I was too quick to suggest using IDF 5.2.2. I'm trying to remember why the build broke when you added

    - name: Setup Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.12'

to build_micropython.yml. The Ubuntu 24.04 runner is supposed to come with 3.12 anyways.

I'll leave it up to you to revert the ci_esp32_idf44_setup and the Python version requirement, if you don't have any better suggestions.

Because ubuntu24.04's own pip package for python installation will report an error, so I do not use it. I've submitted it. Look at the error message

There are three ways to solve the problem

1. use pip --break-system-packages
2. use sudo mv /usr/lib/python3.x/EXTERNALLY-MANAGED /usr/lib/python3.x/EXTERNALLY-MANAGED.bk
3. use actions/setup-python

I think the thrid method are the most reliable.

cc @liamHowatt

[liamHowatt]

Ah right. But even after fixing that, getting IDF v5.2.2 to work will still be a problem. And IDF v4.4 seemed to not work with python 3.12.

[lhdjply]

Ah right. But even after fixing that, getting IDF v5.2.2 to work will still be a problem. And IDF v4.4 seemed to not work with python 3.12.

So I used Python 3.10 version before

[liamHowatt]

We could stick with IDF v4.4, and install both python 3.10 and 3.12 with 3.10 as the default so that the IDF part uses 3.10, and then our own python code micropy_test/__init__.py can run with 3.12. See this.

Is this a good or bad idea?

diff --git a/.github/workflows/build_micropython.yml b/.github/workflows/build_micropython.yml
index 8c1e5e37f..2c793a05d 100644
--- a/.github/workflows/build_micropython.yml
+++ b/.github/workflows/build_micropython.yml
@@ -15,6 +15,12 @@ jobs:
         port: ['unix', 'esp32', 'stm32', 'rp2']
     steps:
     - uses: ammaraskar/gcc-problem-matcher@master
+    - uses: actions/setup-python@v5
+      with:
+        # the last version in the list will be the default. use 3.10 for IDF.
+        python-version: |
+            3.12
+            3.10
     - name: Install Dependencies
       run: |
         sudo add-apt-repository -y "deb http://archive.ubuntu.com/ubuntu `lsb_release -sc` main universe restricted multiverse"
@@ -43,7 +49,7 @@ jobs:
     - name: Setup ESP-IDF
       if: matrix.port == 'esp32'
       run: |
-        source tools/ci.sh && ci_esp32_idf522_setup
+        source tools/ci.sh && ci_esp32_idf44_setup
     - name: Build ESP32 port
       if: matrix.port == 'esp32'
       run: |
@@ -69,13 +75,13 @@ jobs:
       run: make -j $(nproc) -C ports/unix DEBUG=1 LV_CFLAGS="-DMICROPY_LV_USE_LOG=0"
     - name: Install requirements for the test
       run: |
-        python3 -m pip install pillow
+        python3.12 -m pip install pillow
         mkdir lib/lv_bindings/lvgl/tests/micropy_test/artifacts
     - name: Run tests
       if: success() && matrix.port == 'unix'
       run: |
         export XDG_RUNTIME_DIR=/tmp
-        python3 lib/lv_bindings/lvgl/tests/micropy_test/__init__.py --artifact-path=lib/lv_bindings/lvgl/tests/micropy_test/artifacts --mpy-path=ports/unix/build-standard/micropython
+        python3.12 lib/lv_bindings/lvgl/tests/micropy_test/__init__.py --artifact-path=lib/lv_bindings/lvgl/tests/micropy_test/artifacts --mpy-path=ports/unix/build-standard/micropython
 
     - name: Upload Artifacts
       uses: actions/upload-artifact@v4

[lhdjply]

We could stick with IDF v4.4, and install both python 3.10 and 3.12 with 3.10 as the default so that the IDF part uses 3.10, and then our own python code micropy_test/init.py can run with 3.12. See this.

Is this a good or bad idea?

This seems to serve only as a temporary workaround, and I believe it's not recommended. Below is likely the reason for the compilation failure in version 5.2, the include path for esp_adc_cal.h has been changed.

4.4

5.2

So, in this line of code, there should be a conditional check to determine whether to use esp_adc_cal or esp_adc.

cc @liamHowatt What's your opinion on this?

[lhdjply]

cc @liamHowatt Can you look this pr? https://github.com/lvgl/lv_micropython/pull/86

[liamHowatt]

Sorry for not being able to solve the IDF v5.2.2 PR. I tried before you opened that PR, and you got farther than me.

For the current CI issue, the part of the build process in lv_micropython that uses pycparser invokes python as just "python3" which would be the 3.10 installation. The python3.12 -m pip install pycparser line could stay as python3 -m pip install pycparser.

[lhdjply]

For the current CI issue, the part of the build process in lv_micropython that uses pycparser invokes python as just "python3" which would be the 3.10 installation. The python3.12 -m pip install pycparser line could stay as python3 -m pip install pycparser.

It's getting errors and other issues now 😥

I feel like I need to separate the ESP32 from the rest；

[lhdjply]

@liamHowatt It compiles correctly on Ubuntu 22.04, but not on Ubuntu 24.04.

Ubuntu 24.04 still prompts "pip install pycparser."

[lhdjply]

@kisvegabor @liamHowatt The issue with not being able to run 32-bit programs on Ubuntu 24.04 has finally been fixed.

Now there are two CI errors remaining, one for esp32 and one for uefi. I'll send a PR(#6683) for the uefi problem shortly, and could you take a look at the esp32 issue and see if you can help resolve it?

[liamHowatt]

The python that the ESP32 CMake build system is using is the wrong python to run the json generator somehow. I've tried a few different things in my own branch to no avail, I'll come back to it later. The problem is that this CMake package find finds the wrong Python for use here (finds the one that doesn't have pycparser). Locally, CMake seem to choose to use a virtual environment inside the IDF installation. It would be best if we could hint it to use the right one somehow, just from the YAML here.

[kisvegabor]

Quick question: it seems a massive amount of work is going into this update from multiple people. Why is it important to update to Ubuntu 24.04? What will we gain with it?