Add progressive JPEG encoder

[ardabada]

Prerequisites

• [x] I have written a descriptive pull-request title
• [x] I have verified that there are no overlapping pull-requests open
• [x] I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules :cop:.
• [x] I have provided test coverage for my change (where applicable)

Description

This PR adds progressive JPEG encoder (see #10 and #449). Implementation adapted from https://github.com/vstroebel/jpeg-encoder

No tests added yet. Restart interval also should be added. Please take a look if it makes sense to you.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

:white_check_mark: ardabada
:white_check_mark: JimBobSquarePants
:x: Alexandr Ivanov

---

Alexandr Ivanov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[JimBobSquarePants]

Wow! Thanks @ardabada

Tests are failing just now due to a minor StyleCop issue (trailing space) would it be possible to fix that and add some additional unit tests to verify the encoded output?

@br3aker would you be able to help review this? It reuses most of the work you implemented.

[ardabada]

Hi @JimBobSquarePants, I've added a test. The test suite seems quite complicated to me, and I need more time to understand how it works before I can write more complex, byte-level tests.

I've also split the WriteBlock method into two separate methods: WriteDc and WriteAcBlock. This change was necessary because the progressive encoder requires different values in the for loop when writing AC components. Additionally, I've updated the final if statement for writing the end of the block to check runLength > 0 instead of lastValuableIndex, since runLength is reset when encountering a non-zero coefficient. I hope this makes sense. CC @br3aker

I'd like to add a restart interval as well, but it seems to require more changes. WriteMarker is currently in JpegEncoderCore, and it should be called from HuffmanScanEncoder, which isn't an ideal solution. Therefore, I will postpone this for now.

[JimBobSquarePants]

Hi @JimBobSquarePants, I've added a test. The test suite seems quite complicated to me, and I need more time to understand how it works before I can write more complex, byte-level tests.

I've also split the WriteBlock method into two separate methods: WriteDc and WriteAcBlock. This change was necessary because the progressive encoder requires different values in the for loop when writing AC components. Additionally, I've updated the final if statement for writing the end of the block to check runLength > 0 instead of lastValuableIndex, since runLength is reset when encountering a non-zero coefficient. I hope this makes sense. CC @br3aker

I'd like to add a restart interval as well, but it seems to require more changes. WriteMarker is currently in JpegEncoderCore, and it should be called from HuffmanScanEncoder, which isn't an ideal solution. Therefore, I will postpone this for now.

Thanks for the updates! I've no issue with duplicate WriteMarker calls for now if you find it easier to go that way..

I'll pull down your code ASAP and have a good read through. Maybe I can help write tests.

[JimBobSquarePants]

Hi @ardabada apologies for the slow response. The code all looks great so far!

I think you can either

1. Add a duplicate WriteMarkerHeader method to the HuffmanScanEncoder.
2. Make WriteMarkerHeader static and internal passing the stream as a parameter.

I'm happy with whatever approach you take.

For tests I would keep it high level and simply encode/verify the output against expected results.

[JimBobSquarePants]

@ardabada I pulled down your code and added the following test. It appears that writing anything other than the default value causes our decoder to fail.

[Theory]
[WithFile(TestImages.Png.CalliphoraPartial, nameof(NonSubsampledEncodingSetups), PixelTypes.Rgb24)]
[WithFile(TestImages.Png.CalliphoraPartial, nameof(SubsampledEncodingSetups), PixelTypes.Rgb24)]
[WithFile(TestImages.Png.BikeGrayscale, nameof(LuminanceEncodingSetups), PixelTypes.L8)]
[WithFile(TestImages.Jpeg.Baseline.Cmyk, nameof(CmykEncodingSetups), PixelTypes.Rgb24)]
[WithFile(TestImages.Jpeg.Baseline.Ycck, nameof(YcckEncodingSetups), PixelTypes.Rgb24)]
public void EncodeProgressive_CustomNumberOfScans<TPixel>(TestImageProvider<TPixel> provider, JpegEncodingColor colorType, int quality, float tolerance)
where TPixel : unmanaged, IPixel<TPixel>
{
    using Image<TPixel> image = provider.GetImage();

    JpegEncoder encoder = new()
    {
        Quality = quality,
        ColorType = colorType,
        Progressive = true,
        RestartInterval = 7
    };
    string info = $"{colorType}-Q{quality}";

    using MemoryStream ms = new();
    image.SaveAsJpeg(ms, encoder);
    ms.Position = 0;

    // TEMP: Save decoded output as PNG so we can do a pixel compare.
    using Image<TPixel> image2 = Image.Load<TPixel>(ms);
    image2.DebugSave(provider, testOutputDetails: info, extension: "png");

    ImageComparer comparer = new TolerantImageComparer(tolerance);
    image.VerifyEncoder(provider, "jpeg", info, encoder, comparer, referenceImageExtension: "jpg");
}

Here's an encoded jpeg which seems to be decodable by browsers, Windows, and System.Drawing.

And here's how our decoder sees it.

I did find an issue with DRI marker writing where we were writing too many bytes to the stream (see fixed version below)

/// <summary>
/// Writes the DRI marker
/// </summary>
/// <param name="restartInterval">Numbers of MCUs between restart markers.</param>
/// <param name="buffer">Temporary buffer.</param>
private void WriteDri(int restartInterval, Span<byte> buffer)
{
    if (restartInterval <= 0)
    {
        return;
    }

    this.WriteMarkerHeader(JpegConstants.Markers.DRI, 4, buffer);

    buffer[1] = (byte)(restartInterval & 0xff);
    buffer[0] = (byte)((restartInterval >> 8) & 0xff);
    this.outputStream.Write(buffer, 0, 2); // See explicit offset and length.
}

However, I think the issue is with the HuffmanScanDecoder. I've done some debugging and it's finding the markers well enough, perhaps something is not getting reset properly?

@br3aker If you have any time to help out here it would be greatly appreciated.

[ardabada]

Hi, @JimBobSquarePants, sorry for such long silence. I am currently looking into the JpegBitReader, looks like bitstream adjustment to start on the next byte boundary is not handled properly. As a suggestion, we can take restart interval in a separate PR

[JimBobSquarePants]

Hi, @JimBobSquarePants, sorry for such long silence. I am currently looking into the JpegBitReader, looks like bitstream adjustment to start on the next byte boundary is not handled properly. As a suggestion, we can take restart interval in a separate PR

No worries at all and thanks for replying. I'd like to get the bug in the reader fixed if possible before merging so that we don't forget. Would you be happy to investigate?

[ardabada]

Hi @JimBobSquarePants. PR updated. Looks like decoder is fixed now. However i don't really like the duplicated ifs for restart intervals, are you ok with such approach or it's better to keep it in separate methods and track restarts to go in a field?

[JimBobSquarePants]

Legend! Thanks for fixing it. I’ll pull down and review ASAP