pinn
pinn copied to clipboard
procount
WIP: Update buildroot
This is not tested yet. This updates the included buildroot to version 2018.11, the last version with QT4. Differences to the default buildroot 2018.11:
- Add kernel 4.19 support
- Update ca-certificates
- Include PINN packages
- Fixes for the RPi kernel
11,000 changed files! :eyes: Holy moly, that's going to be fun to try and review :wink:
Top tip: When making several changes, as you've done in your list above, it's easiest if each of those changes is in a separate commit, rather than squashing everything into a single commit. That way it's easier to pick out exactly what changes e.g. "Fixes for the RPi kernel" involves.
EDIT: But that's just my opinion of course, @procount may see things differently.
I rebased it, I hope it is okay like this (I can also squash them again if that's too many commits).
I think I found a way to get QT5, I'm still testing it, and many things still need to be done, but I got a working window system:
I got QT5 working, there are still many tweaks needed, but I now got a working build system, and there is one issue: It needs more resources, and can't use the cd (cut-down) firmware. As NOOBS and PINN use the name recovery, I'm not sure if this is okay for you. Also, the RAM usage increases, and PINN is about 20 MB bigger with QT5 (which also uses even more RAM).
@procount @lurch What are your opinions on this?
Note: This works in parts, but there are many things that still need to be done, but before putting even more work into this, I thought I should ask if you even accept these changes.
What's the issue with the cutdown firmware? I've never really studied what the differences were between the different versions.
As I said before, I'd like to move PINN onto a later buildroot and QT5 so that I can use a more up to date version of Ubuntu to develop it on. It's a BIG job and I appreciate your work on this. Hopefully it won't be at the expense of too much bloating, because it is supposed to be small and nimble. Unfortunately, I've not had the time to put any effort into this, but at some point I need to, because I can't/won't integrate anything I don't understand or couldn't support. I therefore need to be able to follow along with what you've been doing when I can devote more time to it. So I would ask you to keep your PRs well documented and clear, with each one performing a specific change, and each change being integratable, so there is always a working version, as far as possible.
I can't guarantee when or if I can integrate these changes - it depends on my other commitments, and how well you progress. So if you want to continue, please do so primarily because YOU want to. I hope PINN continues to be useful, but with other advances like UEFI and devs wanting to use more of UBoot, etc, I do wonder how long it will last. Hopefully long enough to complete my roadmap of new features :smile:
Okay, I'll push more changes soon, I currently have about 20 commits just named "Fix". I currently got a working Window system. Here's a list of all changes to PINN packages I did and I still have to do:
- [x] Use a custom arora with bugfixes like in #411 and QT5 support
- [x] Remove QWS from PINN
- [x] Replace some QT4 features with QT5 features (toAscii was replaced by toLatin1)
- [x] Add a custom compositor
- [ ] Create a way for recovery to change compositor settings
Because the Compositor uses later QT features, but the buildroot currently used here has an older qt5 version, so I need to do the buildroot update & QT5 port in one PR. I can split this into multiple commits, but not create multiple PRs.
I uploaded the version I'm currently working on here. The current version does not boot, there is still something needed, but I need to do a test build first. Also, it currently starts a shell on boot, I will change that later, but this makes debugging easier.
The first commit is just what happens after deleting the buildroot folder and replacing it with buildroot 2020.08-rc1. The commits after that are changes to buildroot which are already done here, and changes that needed to be done to get the latest QT working.
@procount While trying to get this working, I found out that a complete change of the boot partition is needed, which would break the update system, so an update to prepare PINN for the next update would be needed if these changes are implemented (at least if I understand the update system correctly). Alternatively, it would be possible to have the main PINN in a file with another name (like pinn.rfs instead of recovery.squashfs), and recovery.rfs has a custom init that performs all changes needed and reboots, so it can be done in one update.
The change is needed because having the firmware named recovery.elf prevents it from reading fixup.dat, which results in a lot of memory-related errors.
| Old name | New name |
|---|---|
| recovery.elf | start.elf |
| recover4.elf | start4.elf |
| recovery.cmdline | cmdline.txt |
| (NEW FILE) | config.txt |
| (NEW FILE) | fixup.dat |
| (NEW FILE) | fixup4.dat |
| recovery*.img (kernel) | kernel*.img |
Is it not possible to run Qt5 without OpenGL(ES) and a compositor, e.g. just using a simple framebuffer like is used in Qt4?
Actually, only a single window can be used without compositor and OpenGL, which won't work for PINN (browser, language selection, config, ...)
That is surprising. According to https://doc.qt.io/qt-5/qpaintengine.html#details
"Qt provides several premade implementations of QPaintEngine for the different painter backends we support. The primary paint engine provided is the raster paint engine, which contains a software rasterizer which supports the full feature set on all supported platforms. "
Haven't seen that yet. The compositor uses other OpenGL features, but maybe I can get it to work properly.
It seems you are correct. Sounds like this is a backward step by qt in my opinion. Oh well, I'll have to think about the update issue.
@AaronDewes - I've heard reports that Qt5 does not work well with buildroot on the RPi as you have found, mostly probably due to the Rpi specifics themselves. Whilst it may be possible to fix all the issues, this does result in a major change to the PINN software and a lot of risk. Previous attempts to advance PINN in this manner have not been totally successful. It also requires changes to the firmware, buildroot and the PINN recovery program, and extra bloat that may be difficult to manage on a Pi0 or 1.
I wonder if a better approach would be to do this one step at a time. It might be better to concentrate on just evolving buildroot to a later version and reverting to the use of Qt4. This may mean backtracking a bit and porting Qt4 into the new buildroot environment, but it may allow a new development environment to be used a lot sooner. Once that is working properly, converting to qt5 could be reconsidered at a later stage. And maybe some of the issues with it will have been resolved by then.
I've heard reports that Qt5 does not work well with buildroot on the RPi as you have found, mostly probably due to the Rpi specifics themselves.
Most of these issues are fixed with https://github.com/procount/pinn/pull/401/commits/a1ccdd117c54ed055c5c78414251d94ca2d203b4.
It also requires changes to the firmware,
No, with compositor 0.3 (will be pushed soon), the firmware doesn't require any changes.
buildroot
Only adding these patches (https://github.com/procount/pinn/pull/401/commits/a1ccdd117c54ed055c5c78414251d94ca2d203b4), everything else is in 2020.08.
and the PINN recovery program
Only minor changes are needed there (remove some QWS code, and replace it with something else, I'm currently testing a solution for that)
and extra bloat that may be difficult to manage on a Pi0 or 1.
About 20 MB more RAM usage, I don't know if that''s an issue on the 256 MB models (I don't have one), but I don't think too many problems are caused there.
It might be better to concentrate on just evolving buildroot to a later version and reverting to the use of Qt4.
I've tried that, but haven't been successful. I'll try again if my QT5 attempt fails, but as I said before, compositor 0.3 is much better.
But you're right, QT5 is really broken on RPi, but I'm trying to change that: https://codereview.qt-project.org/c/qt/qtwayland/+/310822.
Sorry for not working on this, I had some issues, because I didn't get fixup.dat to work, because I didn't know it had to be called fixup_rc.dat, but NOOBS 3.5 resolved that issue, I'll rebase and update this PR after you merged NOOBS 3.5 (I assume you'll do that, If not, please tell me), I'll wait with rebasing because otherwise I'd have to do the rebase again.
No worries at all. I appreciate all your efforts and hope it will work out in the end. Timescales are all yours. 😉 Yes I intend to merge the noobs 3.5 changes. Hopefully soon, but not sure when.
I'll wait for Buildroot 2020.02 to release so I can directly update this to Qt 6, because QT 6 doesn't require any major changes from QT 5, but still needs some optimizations