rockchip64-5.18: add LibreELEC DRM/V4L2 patches.

[AdamLantos]

Description

These patches exist on rockchip-5.18, but not on rockchip64-5.18.

Patches that are part of LibreELEC patchset are dropped. Rest of the patches adjusted to apply cleanly. The only patch that gave me trouble was rk356x-vop2-support.patch due to multiple concurrent changes to dw_hdmi-rockchip.c. The patch now applies and compiles, and works on RockPi4-b, but I don't have a RK35xx device to test with, so any tests with such hardware is appreciated!

How Has This Been Tested?

• [x] Compiled with mainline v5.18.3. Patches apply cleanly.
• [x] Booted on RockPi4-b.
• [x] Tested HDMI/rkvdec on RockPi4-b with FFMpeg-v4l2-request patchset and manually built Kodi and Moonlight (H.264 and HEVC hw decoding on 4k@60Hz).

Checklist:

• [ ] My code follows the style guidelines of this project
• [x] I have performed a self-review of my own code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [x] My changes generate no new warnings
• [x] Any dependent changes have been merged and published in downstream modules

[paolosabatino]

Hello, I maintain the rockchip LE patches for rockchip and rk322x families. I'm glad about this proposal, but I guess there should be some discussion with @armbian/boards-rockchip maintainers if this is wanted. In my opinion, these patches cause no harm and I saw a nice improvement on a rk3328-class device in desktop performance.

I see, and correct me if I'm wrong, that they are copied as-is from rockchip family; a good idea is to have the very same LE patchset with same filenames for all the three families (rk322x, rockchip and rockchip64) so modifications are easy to propagate and, from time to time, keep them aligned to LE patchset. Other patches are built upon these LE patches like this patch, which I suggest to bring in as well (prefers RGB HDMI modes over YCbCr, fixing sluggish red-on-black characters)

Let's wait for other opinions.

[AdamLantos]

Hi Paolo,

I'm glad about this proposal, but I guess there should be some discussion with @armbian/boards-rockchip maintainers if this is wanted.

Agreed. Sorry I am very new to Armbian processes, so I figured I'd start with doing the work and then solicit opinions. I have been running these patches on my RockPi4 for weeks now and have good experience with Kodi and Moonlight, finally able to stream in 4k HDR is a big plus :-)

I see, and correct me if I'm wrong, that they are copied as-is from rockchip family; a good idea is to have the very same LE patchset with same filenames for all the three families (rk322x, rockchip and rockchip64) so modifications are easy to propagate and, from time to time, keep them aligned to LE patchset. Other patches are built upon these LE patches like this patch, which I suggest to bring in as well (prefers RGB HDMI modes over YCbCr, fixing sluggish red-on-black characters)

These were copied from the Armbian rockchip-5.18 patches, but I rebased them on top of mainline v5.18.3 as there were a few minor changes upstream that shifted the patches by a few lines. It would definitely be great if this patchset was maintained in a single place for all rockchip families, and I am happy to adjust this PR as needed / discussed here.

Thanks!

[iav]

side note: add-csgpio-to-rockchip-spi.patch breaks current compilation for rockchip64.

drivers/spi/spi-rockchip.c:767:12: error: redefinition of ‘rockchip_spi_setup’
make[2]: *** [scripts/Makefile.build:288: drivers/spi/spi-rockchip.o] Error 1
make[1]: *** [scripts/Makefile.build:551: drivers/spi] Error 2

[AdamLantos]

I have no issues with accepting LE patches, HOWEVER:

• The patches must have a name that indicates what they change, and these are massive unmanageable agglomerates mashed into a single file that some individuals find sufficient for their personal workflow. This is fine in their own respective project. Here they must be split up.
• Notice the naming convention used in the patches you deleted. There is a reason for it, so that a team can debug/easily determine what patches are touching what devices. Please adjust proposed patches accordingly to conform.

Full disclosure: I am not a maintainer in either LE or Armbian, but from what I've seen in these patches, the LE patchset is in a mbox format with individual patches and descriptions, which allow it to be very easily rebased (git am). Compared to some patches like the vop2 patch in armbian which is just one big git diff.

While I agree that the LE set is somewhat arbitrarily split into several big patch files, but I don't think hunting down individual patches in it would be significantly worse than the current patches in rockchip64 (some of which don't even compile without patches that are applied later).

I am happy to make slight adjustments to how this laid out, but splitting out the LE patches into logical smaller files is much more than I can handle at this point, and I am deferring decisions on this to Armbian maintainers.

I know this is a pain in the ass, but a bigger one is the now 5 times repeated situation where the patchset becomes pure entropy and people apply multiple different fixes in similarly nondescript patch names and someone (historically me) has to spend over a week going line by line trying to figure out why the entire family suddenly doesn't have I2S or eMMC support. (This has been a recurring theme due to the addition of vendor and LE patches in Meson64 and here in Rockchip64)

Too many cooks in this kitchen to allow this to be merged without changes. Paolo basically maintains the rockchip family on his own, so I haven't complained as the Tinkerboard is better in his hands these days. This family (rockchip64) has too many devices, SoC's and maintainers to add these massive blobs.

As someone who spent 3 days trying to sort through this, I can't help but agree with the number of different patches being way too high. Especially the situation with rk356x vop2 set, which I haven't found anywhere else. It would be extremely beneficial to have some comments on where these patches are pulled from (Collabora, linux-next, or somewhere else) and instructions how the most recent versions could be re-pulled.

[amazingfate]

I have no issues with accepting LE patches, HOWEVER:

• The patches must have a name that indicates what they change, and these are massive unmanageable agglomerates mashed into a single file that some individuals find sufficient for their personal workflow. This is fine in their own respective project. Here they must be split up.
• Notice the naming convention used in the patches you deleted. There is a reason for it, so that a team can debug/easily determine what patches are touching what devices. Please adjust proposed patches accordingly to conform.

Full disclosure: I am not a maintainer in either LE or Armbian, but from what I've seen in these patches, the LE patchset is in a mbox format with individual patches and descriptions, which allow it to be very easily rebased (git am). Compared to some patches like the vop2 patch in armbian which is just one big git diff.

While I agree that the LE set is somewhat arbitrarily split into several big patch files, but I don't think hunting down individual patches in it would be significantly worse than the current patches in rockchip64 (some of which don't even compile without patches that are applied later).

I am happy to make slight adjustments to how this laid out, but splitting out the LE patches into logical smaller files is much more than I can handle at this point, and I am deferring decisions on this to Armbian maintainers.

I know this is a pain in the ass, but a bigger one is the now 5 times repeated situation where the patchset becomes pure entropy and people apply multiple different fixes in similarly nondescript patch names and someone (historically me) has to spend over a week going line by line trying to figure out why the entire family suddenly doesn't have I2S or eMMC support. (This has been a recurring theme due to the addition of vendor and LE patches in Meson64 and here in Rockchip64) Too many cooks in this kitchen to allow this to be merged without changes. Paolo basically maintains the rockchip family on his own, so I haven't complained as the Tinkerboard is better in his hands these days. This family (rockchip64) has too many devices, SoC's and maintainers to add these massive blobs.

As someone who spent 3 days trying to sort through this, I can't help but agree with the number of different patches being way too high. Especially the situation with rk356x vop2 set, which I haven't found anywhere else. It would be extremely beneficial to have some comments on where these patches are pulled from (Collabora, linux-next, or somewhere else) and instructions how the most recent versions could be re-pulled.

I added the rk356x-vop2-support.patch, which is original from https://patchwork.kernel.org/project/linux-rockchip/list/?series=634531, which is half merged in 5.19 rc1. This patch will get a lot smaller when we update to 5.19. I can use kodi with drm prime support on armbian without these libreELEC patches now. I don't know what these patches has imporoved technically, but I will try it.

[lanefu]

Test images [will add more] here:

Armbian_22.08.0-trunk-PR-3893_Rock-3a_bullseye_edge_5.18.3.img.xz Armbian_22.08.0-trunk-PR-3893_Rock-3a_jammy_edge_5.18.3_xfce_desktop.img.xz

[amazingfate]

Test images [will add more] here:

Armbian_22.08.0-trunk-PR-3893_Rock-3a_bullseye_edge_5.18.3.img.xz

Kodi needs panfrost driver to use drm, which should be not supported on bullseye. Jammy should be better.

[AdamLantos]

This patch will get a lot smaller when we update to 5.19. I can use kodi with drm prime support on armbian without these libreELEC patches now. I don't know what these patches has imporoved technically, but I will try it.

Thank you Jianfeng! I needed the LE patches in order to get HEVC 10-bit HDR hardware accelerated. H.264 was working with standard Armbian with https://github.com/jernejsk/FFmpeg/commits/v4l2-request-hwaccel-4.4 patchset, but HEVC was not. Although there is a slight adjustment I needed to make to FFMpeg to allow 32 slices at https://github.com/jernejsk/FFmpeg/blob/v4l2-request-hwaccel-4.4/libavcodec/v4l2_request_hevc.c#L24 -- I'm not sure if that would work on all boards or not.

I see the mainline 5.19 tree should merge the majority of GPU changes, but I'm not sure V4L2 HEVC support would make it since it's still being worked on at https://patchwork.kernel.org/project/linux-media/list/?series=645606. So we could either apply the current LE patches to get the WIP version working, or just wait until everything is merged upstream. Perhaps everything will be lined up in 5.20? So we should not need to keep these patches forever.

[amazingfate]

This patch will get a lot smaller when we update to 5.19. I can use kodi with drm prime support on armbian without these libreELEC patches now. I don't know what these patches has imporoved technically, but I will try it.

Thank you Jianfeng! I needed the LE patches in order to get HEVC 10-bit HDR hardware accelerated. H.264 was working with standard Armbian with https://github.com/jernejsk/FFmpeg/commits/v4l2-request-hwaccel-4.4 patchset, but HEVC was not. Although there is a slight adjustment I needed to make to FFMpeg to allow 32 slices at https://github.com/jernejsk/FFmpeg/blob/v4l2-request-hwaccel-4.4/libavcodec/v4l2_request_hevc.c#L24 -- I'm not sure if that would work on all boards or not.

I see the mainline 5.19 tree should merge the majority of GPU changes, but I'm not sure V4L2 HEVC support would make it since it's still being worked on at https://patchwork.kernel.org/project/linux-media/list/?series=645606. So we could either apply the current LE patches to get the WIP version working, or just wait until everything is merged upstream. Perhaps everything will be lined up in 5.20? So we should not need to keep these patches forever.

AFAIK rk35xx doesn't suport HEVC because it has a new vpu and the driver is not implemented yet. I've just regenerated the rocckchip64 patches for 5.19 rc1, here is the source: https://github.com/amazingfate/build/tree/rockchip64-5.19. I'm going to update rockchip64 edge to 5.19 after kernel 5.19 is released, So if these patches are merged there will be some work for 5.19 to get these patches working, and there may also be work on 5.20.

[paolosabatino]

Well, splitting the patches into smaller ones may be useful for general maintainability when patches got mainlined in vanilla kernel, so they are easier to be deleted, edited or "disabled" for further inspection in armbian; I still could not find a valid utility that splits the patches from an mbox format without losing the email header and body (maybe git am -> git format-patch ? Never tried...)

Surely it would be useful to put them in a separate directory than mixing all of them, so when LE people does a rebase, it is just a matter of deleting the existing directory content and splitting again the monolithic LE patches to keep armbian and LE repo aligned.

This does not solve the sudden disappear of the dtb node for this or that device as @Tonymac32 said. It looks to me that LE patches just get "rebundled" by LE people from patchwork.kernel.org in mbox format from time to time and I had no great surprises updating them on rk322x and rockchip targets. It is worth saying that those targets are very stable and there are a total of 1 officially supported, 1 EOL and 2 CSC devices there, not the plethora of rk3399/rk3328.

A good thing that comes handy is the pull request with approval barrier we have right now: this forces people to give a seal of approval, sharing responsibility, and thus having some code review for the patches and not blindly kicking everything into.

Indeed if a LE patches directory would exist, a rule should be clear: every modification in that directory will be lost on next alignment because the content will be purged and rebuilt from LE monolithic patches from time to time.

[AdamLantos]

Well, splitting the patches into smaller ones may be useful for general maintainability when patches got mainlined in vanilla kernel, so they are easier to be deleted, edited or "disabled" for further inspection in armbian; I still could not find a valid utility that splits the patches from an mbox format without losing the email header and body (maybe git am -> git format-patch ? Never tried...)

Since these patches are ordered alphabetically, the LE patchset is always the first that gets applied. The patchset could be rebased on top of mainline very easily. For each patch: apply on top of v5.18.3 with git am <patchfile>, then rebase it on the new mainline with git rebase <new tag>, then git format-patch v5.18.3..<new tag> --stdout > <patchfile>. That should take care of any upstreamed patches. Obviously ones that were upstreamed with diffs will need to be manually merged, there is just no way around that.

Surely it would be useful to put them in a separate directory than mixing all of them, so when LE people does a rebase, it is just a matter of deleting the existing directory content and splitting again the monolithic LE patches to keep armbian and LE repo aligned.

+1 to this. Ideally it would be possible to build kernels with or without the extra set of LE patches (e.g. for a server build we shouldn't need LE patches, but for desktop or media centers it's very useful). However since we have patch conflicts between LE and existing Armbian patches, that's unrealistic unless someone wants to spend weeks to refactor the entire patchset so the "media-centric" patches could be applied individually.

[lanefu]

Test images [will add more] here: Armbian_22.08.0-trunk-PR-3893_Rock-3a_bullseye_edge_5.18.3.img.xz

Kodi needs panfrost driver to use drm, which should be not supported on bullseye. Jammy should be better.

Added Jammy xfce

[Tonymac32]

Since these patches are ordered alphabetically, the LE patchset is always the first that gets applied.

Until they are not. This is a consequence of no one following a convention, you can not assume your personal patches will be the first ones applied. Let's say the next contributor wants to add their favorite Arch kernel patches, or FriendlyElec, or Hardkernel patches, and similarly don't want to be bothered organizing them, so they start with "001" or some such nonsense.

[amazingfate]

tested on my rock-3a, no other side effects, everything works

[igorpecovnik]

I would just note that build framework support patch series which allows getting things in order. Perhaps this way @Tonymac32 ?

Example: https://github.com/armbian/build/tree/master/patch/kernel/archive/sunxi-5.15

[AdamLantos]

I would just note that build framework support patch series which allows getting things in order. Perhaps this way @Tonymac32 ? Example: https://github.com/armbian/build/tree/master/patch/kernel/archive/sunxi-5.15

Thanks Igor! That looks promising. If we can get an agreement regarding the desired layout, I can try adapting this PR to the series.conf style by separating LE and Armbian patches into their respective directories.

I have a question - who needs to approve this PR ultimately? So far @paolosabatino, @Tonymac32, @amazingfate, @lanefu, @iav chimed in. Is it fair to say that the biggest flag raised was the layout of patches by @Tonymac32, and if that discussion gets resolved, this can get merged?

[lanefu]

Yeah If @Tonymac32 and @paolosabatino sign off should be fine to merge

[lanefu]

@AdamLantos thanks for being flexible

[paolosabatino]

I guess the first step is to put the LE and Armbian patches into separate directories, this is a step to keep them tidy. The issues raised by @Tonymac32 however still holds, but surely can be managed in a easier manner if the patches are in separate directories are easier to review.

If @Tonymac32 does not agree - which is a perfectly reasonable position, as long as in the past changes in LE patches broke things - I guess we just need to find another solution (maybe not importing the whole LE patchset but just DRM/V4L2 related ones, which are already separated by LE folks).

In the meantime I think we can wait for @AdamLantos to split and separate LE patches into own directory.

[The-going]

A very interesting discussion. If a decision is made to switch to using a series of patches, a manual tool that I use may be needed. This script makes it easier to work.

[jernejsk]

Just FYI, LE patches for video decoding will be soon obsolete, at least most of them. A lot of work is going on to stabilize HEVC and other improvements, like VP9 10-bit support (5.20 material).

[AdamLantos]

OK, so I tried moving the patches into specific subdirectories (patches.libreelec and patches.armbian) and introduced a series.conf file. But I run into an issue: unfortunately now a bunch of patches just straight don't apply because the armbian script processes these patches differently when a series.conf file is used.

When individual patch files are applied, the script deletes kernel source files that are recreated by patches at lib/compilation.sh#L939-L940, but we don't do that when series.conf style of patching is used at lib/compilation.sh#L973-L977. I suspect this latter would actually be the better approach, because right now we just silently overwrite whatever is upstream, instead of telling board maintainers that the patch files are out of sync with the mainline.

Example: add-board-helios64.patch overwrites arch/arm64/boot/dts/rockchip/rk3399-kobol-helios64.dts, but this patch and any downstream patches to these files are not properly applied.

I probably won't have time to re-do these patches, so I'll wait for 5.19 to be cut and Armbian patches adjusted before I try this again.

[The-going]

When individual patch files are applied, the script deletes kernel source files that are recreated by patches at https://github.com/armbian/build/blob/6bf936d2e3eeae8cddadb294ead89d0e1ce89c7d/lib/compilation.sh#L939-L940

To put it mildly, this is "non-standard and not explicit behavior."

Usually we wait for the standard behavior in applying patches.

The idea is that you just make a branch in your local kernel storage that will track the remote linux-5.18.y branch:

[remote "origin"]
        url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
        fetch = +refs/heads/linux-5.18.y:refs/remotes/origin/linux-5.18.y

Apply all your patches as git am or make your own commits and then extract them to the target folder patch/kernel/archive/rockchip64-5.18/patches.name with this script.

[AdamLantos]

When individual patch files are applied, the script deletes kernel source files that are recreated by patches at https://github.com/armbian/build/blob/6bf936d2e3eeae8cddadb294ead89d0e1ce89c7d/lib/compilation.sh#L939-L940

To put it mildly, this is "non-standard and not explicit behavior."

It is surprising, yes, but it's something that Armbian's own script does and the majority of boards happen to use this broken behavior with a list of .patch files, which leads to silent deletion of upstream files. Only sunxi family appears to be using the series.conf file which leads to proper detection of these patching conflicts.

The idea is that you just make a branch in your local kernel storage that will track the remote linux-5.18.y branch:

[remote "origin"]
        url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
        fetch = +refs/heads/linux-5.18.y:refs/remotes/origin/linux-5.18.y

Apply all your patches as git am or make your own commits and then extract them to the target folder patch/kernel/archive/rockchip64-5.18/patches.name with this script.

Right, but this PR was about adding LibreELEC patches to rockchip64, and now it's about revising existing rockchip64 patches, and perhaps even fixing this bit of Armbian core code to stop deleting upstream files in the kernel source: https://github.com/armbian/build/blob/6bf936d2e3eeae8cddadb294ead89d0e1ce89c7d/lib/compilation.sh#L939-L940

[The-going]

I first added only megous patches to the series: https://github.com/armbian/build/tree/master/patch/kernel/archive/sunxi-5.10 They will be applied first. I had to fix several patches coming after them that added new files. Now these patches fix existing files. These files did not exist before, but time passed and they appeared in the upstream. Now both patching mechanisms are working here.

The choice depends only on you. I'm just trying to help as much as I can.

[The-going]

Armbian core code to stop deleting upstream files in the kernel source:

https://github.com/armbian/build/blob/6bf936d2e3eeae8cddadb294ead89d0e1ce89c7d/lib/compilation.sh#L939-L940

There is no need to change anything here. This will affect other families and will add to all the work of rework.

[lanefu]

yay....did we talk a PR to death?

[paolosabatino]

Well importing LE patches brings also many changes to existing patchset, but also there seems not to be an agreement about how patches should be named... Personally I understand both the concerns from @piter75 and @150balbes and I wonder why they can't be synthesized into a joint proposal for patches like:

class-sequence-description.patch

we can define a closed set of one-word patch classes (like driver, board, general, overlays, core, misc, ...) and among them they are strictly applied by a three-digits sequence number.

Obviously this does not solve all the problems, but at least it could be a starting point for sane cooperation avoid lengthy discussions that can bore patient and welcomed contributors like @AdamLantos

[mo123]

@AdamLantos Hi, can you share your compiled ffmpeg and Kodi .deb files and how to install it? I tried these Kodi and ffmpeg(from here - https://forum.radxa.com/t/app-kodi-on-armbian-with-hardware-accelerating-decode/10170) but Kodi could only play 1080p H264, HEVC was still software decoded with 100% CPU usage on my RockPi 4B with Armbian I build with this PR. Do you need to build Armbian with multimedia support or without to make the custom ffmpeg work correctly? I build Armbian with multimedia support and it asked if I want to replace ffmpeg with older ones, so probably something went wrong. Do you need to run mpv or Kodi with special parameters?

[igorpecovnik]

that can bore patient and welcomed contributors like

@AdamLantos I hope we didn't scare you :) ? Anything I can help to make it in move?