GaussianImage icon indicating copy to clipboard operation
GaussianImage copied to clipboard

Byte calculation on analysis wo ec missing Cholesky Channels multiplier

Open Downchuck opened this issue 5 months ago • 6 comments

In the gaussianimage_cholesky.py we have a six bit quantizer with three channels:

            self.cholesky_quantizer = UniformQuantizer(signed=False, bits=6, learned=True, num_channels=3)

In the analysis_wo_ec method, the number of channels are missing in the bit measurement:

        total_bits += quant_cholesky_elements.size * 6 #cholesky bits 

and

        cholesky_bits += len(quant_cholesky_elements) * 6

I believe that those should be multiplied by 3.

Downchuck avatar Aug 26 '24 02:08 Downchuck