uploader: compress with zstd

[sshane]

[github-actions[bot]]

This PR has had no activity for 14 days. It will be automatically closed in 3 days if there is no activity.

[adeebshihadeh]

If you add to the milestone, it should be exempt from stale.

[sshane]

@adeebshihadeh The last blocker is the openpilotci and related sources for when we add future segments, they're hard-coded to check for rlog.bz2 URLs. Are you fine with merging this now, and we fix it up after, or do you want this fixed first?

I would imagine there's a few more unknown cases that will pop up that we'll have to fix along the way (shouldn't be show-stopping), so I'd prefer getting this long-running project finally merged to finish it up faster, and I can then work in parallel on those issues.

---

According to top, I'm seeing an increase from 0.2% memory usage with bz2 to 1.4% with zstd compressing an rlog at level 10. Level 15 brings it to ~2.8%, level 19 brings it to ~3.2% on a 3X (all stable). Both bz2 and zstd pin a core to 100% usage. In power save mode, zstd compresses an rlog in ~6s, bz2 is ~11s.

I tried increasing the level to 15, but it ~2.5x the time and produced no benefits at all for a few qlogs I tried (same exact size), but does shave down ~0.3MB off the rlogs.

[adeebshihadeh]

I'm fine with merging now.

Can we change the zstd level in the future by just changing the value here? Or does it require changes on the decompression side?

[sshane]

Nope, can be changed to anything.