deflate, Z_SYNC_FLUSH, and double trailer emission

[basiliscos]

I think I've something, which resembles bug in zlib. The full source is available at [1]. The issue description is:

when it does deflate() (with Z_SYNC_FLUSH) the "Hello" world into 12-byte buffer, it returns 0, and everything is OK, as the tx_stream.avail_out > 0, saying it consumed 11 bytes. It can be transmitted over network.

However, when it does deflate() into 11-byte buffer, it also returns 0, and as tx_stream.avail_out 0, and according to the docs:

If deflate returns Z_OK and with zero avail_out, it must be called again after making room in the buffer because there might be more output pending.

I invoke it once more with newly allocated buffer, and, to my surprice, it outputs something into the buffer again. Here is a excempt from GDB:

(gdb) p out1 $1 = "\362H\315\311\311\a\000\000\000\377\377" (gdb) p out2 $2 = "\362H\315\311\311\a\000\000\000\377\377\000\000\000\377\377", '\000' <repeats 33 times>

As I can see, the trailer is emitted once more. And, as we use Z_SYNC_FLUSH in our webscoket implementation, we cut the trailer, but in the output buffer 2 it is cut not enough, and it leads to protocol corruption. What can I do? It was my assumption, that it should output nothing in the output buffer 2, to let the outputs be identical.

Used software: zlib-1.2.11, linux 5.8.16 (void linux distrubution). The same happens on debian.

Could you help, please?

[1] https://gist.github.com/basiliscos/ac6f050abdd2746d5114ec2ce2b30cea

[minchai23]

I think you should call deflate with Z_FINSH when you don't have input data.

[madler]

Due to the design of the zlib interface, it unfortunately cannot tell that you are not asking for another flush marker when you call again with Z_SYNC_FLUSH and it has provided the previous flush marker in its entirety. To avoid this issue, you need to make sure that there is enough output space, plus one, to complete the flush when requesting it. One way to do that is to complete the previously supplied data with Z_BLOCK and to consume the resulting output until avail_out is not zero. Then request a Z_SYNC_FLUSH (or Z_FULL_FLUSH) with a fresh output buffer that is at least six bytes in length.