Wrong decoded result on high frequency textures with cheng2020-anchor

[chenhsiu48]

Bug

To Reproduce

Steps to reproduce the behavior:

1. encode this image using codec.py encode --model cheng2020-anchor -q 6
2. wrong decoded results on high frequency textures

Expected behavior

The high frequency textures should be correct.

Environment

Additional context

[chenhsiu48]

model cheng2020-attn has similar issue. just not that severe.

[YodaEmbedding]

That looks like subpixel convolution upscaling gone wrong.

I believe @chyomin06 has run into similar issues. I suspect it's because the encoder/decoder are not "in sync" due to non-deterministic CUDA operations. Perhaps try running it with CPU-only to see if it works correctly.

[chenhsiu48]

@YodaEmbedding the CPU mode has the same issue

[YodaEmbedding]

My other theory is that the image lies far outside of the distribution of images (dataset) that it was trained on, so each of the 4 upscales within the synthesis transform $g_s$ is generating lots of artifacts that lie outside each successive layer's "typical" input distribution. This leads to a compounding effect in the error as the latent tensor gets upscaled.

Possible strategies to prevent this (with training required):

• augmenting the dataset with more high frequency or noisy samples
• some sort of regularization
• ICNR initialization to start with non-artifactful filters, rather than hoping the input dataset provides enough "adversarial" examples
• methods that constrain/bound/ensure well-behaved distributions (e.g. clipping, normalizing flows, ...)

[chenhsiu48]

@YodaEmbedding Thanks for the clue. You're right. The released model wasn't trained with enough high frequency samples so caused that problem. This issue is fixed after I re-train my model.

Since many researchers build their work on top of CompressAI, this issue in the released model may affect their work. I can help to re-train those models. To be compatible with what you have done, I need the lambda values for quality 1 to 6 of cheng2020-anchor and cheng2020-attn. Where can I find it in the repo? Anything else I need to pay attention to?

[chenhsiu48]

@YodaEmbedding Never mind. I get it from https://interdigitalinc.github.io/CompressAI/zoo.html#training

I need the lambda values for quality 1 to 6 of cheng2020-anchor and cheng2020-attn