Cherry-pick upstream patches to 5.15.y for ARM64

[parheliamm]

Upstream has some performance patches should be cherry-picked for better performance.

1. CRC32 patches.
2. updated ARM optimized str{n}cmp patches.
3. ext4 patches

All of them can be applied directly without any changes.

[popcornmix]

Thanks for this. Can you provide and timing numbers before and after this change that shows the improvement?

[parheliamm]

Frankly, most of the patches have benchmark test results on their commit message. So I didn't reproduce the result on RPI4. I cherry-picked these patches to my branch and did a sanity test, there are no issues found.

BTW, this is my first time submitting PR to RPI4, I am not sure about the WoW on RPI4. I will follow the WoW in the future and would like to submit more patches to improve RPI4 performance.

[popcornmix]

We are preparing for a kernel bump in apt, so this won't merge until that's out of the way.

There is some risk to a PR like this. We are testing a combination of commits that hasn't been tested by others. There is a small chance that a commit here, that is completely safe, on, say 5.19 (due to other structural changes) could introduce an issue on 5.15. When code affects the ext4 driver, then even a very small chance of corruption is concerning.

We'd also like to know what the real world benefits are. While a very artificial benchmark (on another platform) of

echo /run/riteshh/mnt/{1..64} |sed -E 's/[[:space:]]+/ -d /g' \
  | xargs -I {} bash -c "sudo fs_mark -L 100 -D 1024 -n 1024 -s0 -S5 -d {}"

may go faster, is there anything a Pi user can actually see (e.g. quicker launching of chromium, or faster apt install of a package?)

I'll pick these commits into a system I'm using just to check I see nothing bad. If anyone else can test and report that would be helpful.

[parheliamm]

Thanks for your explanation. I will try to benchmark it on my side. On the other hand, I recognized you plan to bump the kernel version, I am not sure it has the same policy as the cherry-pick policy? In other word, if you plan to bump the kernel to 5.18, did you plan to evaluate the new kernel version performance? Per my understanding, I don't prefer to bump the kernel version frequently.

BTW, current pi-os didn't enable fast commit on ext4 partition, if you would like to try fast_commit, please refer to below cmd: tune2fs -O fast_commit /dev/drivepartition

[popcornmix]

Bumping to a stable upstream kernel release (e.g. 5.18) is testing a known set of commits (that has been peer reviewed by experts in the area).

The issue that can occur is where there are connected patch series. e.g. a dev discovers that optimising code, exposes a race condition (that was always present, but not triggered with the old timing). We get two patch series submitted. The first fixes the race condition (which was always a theoretical bug). Once merged he can submit the actual optimisation.

Now, just cherry-picking the optimisation to an older kernel tree may be unsafe.

Sometimes commits do get back-ported upstream (usually only bug fixes), but that will only occur with an ack from the code's maintainer who will understand if specific commits are safe on older trees.

Personally, I'd imagine the strcmp and crc32 commits are pretty likely to be self contained and safe (an inspection of the differences between 5.15 with the patch and a newer upstream version of the file would give more confidence).

But ext4 is likely to be a far more complicated scenario, with dependencies on many other parts of kernel, that I'm not in a position to say it will be safe. If the cherry pick came from the author of the patch or a maintainer then we'd know it was considBut ext4 is likely to be a far more complicated scenario, with dependencies on many other parts of kernel, that I'm not in a position to say it will be safe. If the cherry pick came from the author of the patch or a maintainer then we'd know it was considered safe by an expert.ered safe by an expert.

But without some confirmation that the performance benefit is worthwhile to an average pi user, then it's unlikely to be worth even minimal risk. But if, say, an apt install is 10% faster after the patch, then we'd become a lot more interested.

[parheliamm]

Bumping to a stable upstream kernel release (e.g. 5.18) is testing a known set of commits (that has been peer reviewed by experts in the area).

The issue that can occur is where there are connected patch series. e.g. a dev discovers that optimising code, exposes a race condition (that was always present, but not triggered with the old timing). We get two patch series submitted. The first fixes the race condition (which was always a theoretical bug). Once merged he can submit the actual optimisation.

Now, just cherry-picking the optimisation to an older kernel tree may be unsafe.

Sometimes commits do get back-ported upstream (usually only bug fixes), but that will only occur with an ack from the code's maintainer who will understand if specific commits are safe on older trees.

Personally, I'd imagine the strcmp and crc32 commits are pretty likely to be self contained and safe (an inspection of the differences between 5.15 with the patch and a newer upstream version of the file would give more confidence).

But ext4 is likely to be a far more complicated scenario, with dependencies on many other parts of kernel, that I'm not in a position to say it will be safe. If the cherry pick came from the author of the patch or a maintainer then we'd know it was considBut ext4 is likely to be a far more complicated scenario, with dependencies on many other parts of kernel, that I'm not in a position to say it will be safe. If the cherry pick came from the author of the patch or a maintainer then we'd know it was considered safe by an expert.ered safe by an expert.

But without some confirmation that the performance benefit is worthwhile to an average pi user, then it's unlikely to be worth even minimal risk. But if, say, an apt install is 10% faster after the patch, then we'd become a lot more interested.

Thanks for your explanation, I got your point at this moment. Most of your explanation is acceptable per my perspective. If the rpi-os support F2FS or XFS instead of EXT4, EXT4 related patches is useless at that time. As you might be knwo, EXT4 random R/W didn't has advantage compare with XFS/F2FS, especially RPI4 IO performance is bottle neck even though switch to SSD.

[parheliamm]

Bumping to a stable upstream kernel release (e.g. 5.18) is testing a known set of commits (that has been peer reviewed by experts in the area).

The issue that can occur is where there are connected patch series. e.g. a dev discovers that optimising code, exposes a race condition (that was always present, but not triggered with the old timing). We get two patch series submitted. The first fixes the race condition (which was always a theoretical bug). Once merged he can submit the actual optimisation.

Now, just cherry-picking the optimisation to an older kernel tree may be unsafe.

Sometimes commits do get back-ported upstream (usually only bug fixes), but that will only occur with an ack from the code's maintainer who will understand if specific commits are safe on older trees.

Personally, I'd imagine the strcmp and crc32 commits are pretty likely to be self contained and safe (an inspection of the differences between 5.15 with the patch and a newer upstream version of the file would give more confidence).

But ext4 is likely to be a far more complicated scenario, with dependencies on many other parts of kernel, that I'm not in a position to say it will be safe. If the cherry pick came from the author of the patch or a maintainer then we'd know it was considBut ext4 is likely to be a far more complicated scenario, with dependencies on many other parts of kernel, that I'm not in a position to say it will be safe. If the cherry pick came from the author of the patch or a maintainer then we'd know it was considered safe by an expert.ered safe by an expert.

But without some confirmation that the performance benefit is worthwhile to an average pi user, then it's unlikely to be worth even minimal risk. But if, say, an apt install is 10% faster after the patch, then we'd become a lot more interested.

Thanks for your explanation, I got your point at this moment. Most of your explanation is acceptable per my perspective. If the rpi-os support F2FS or XFS instead of EXT4, EXT4 related patches is useless at that time. As you might be knwo, EXT4 random R/W didn't has advantage compare with XFS/F2FS, especially RPI4 IO performance is bottle neck even though switch to SSD.

[lategoodbye]

Commit "ext4: make mb_optimize_scan performance mount option work with extents" has been reverted because of #5097 while the real issue is still under investigation upstream. So applying it again requires a good explanation ;-)

[parheliamm]

Commit "ext4: make mb_optimize_scan performance mount option work with extents" has been reverted because of #5097 while the real issue is still under investigation upstream. So applying it again requires a good explanation ;-)

I double check the #5097, it is interesting, I didn't see the relationship between reproducing step with the patch. I will try it on my raspi4. The different scenarios might be as below:

1. My OS is 64bit.
2. My rootfs is under SSD. I will try my side and report the result soon or later.

[lategoodbye]

Yes, the mentioned commit is not the real issue, but it's a easy workaround to avoid the performance regression caused by mb_optimize_scan.