Color-arithmetic should _probably_ be done in linear-color space and not sRGB
I believe right now all image-arithmetic is operating upon image color values that are within the non-linear sRGB color-space. So operations upon them like sums and averages and conversions may not be indicative to the actual underlying linear color values and may have to be converted from sRGB into linear color space before producing the DCT, and then back into sRGB when intended to be presented to the user.
Ex, sections of code like here should probably be converting the sRGB pixel data into linear before converting into LPQA: https://github.com/evanw/thumbhash/blob/df761ae52c218d22192bd6aee380897f5a887218/rust/src/lib.rs#L41-L51
The default behavior for many image-loading libraries is to provide image-data in the sRGB color-space with no conversions as well.