Chronicle-Queue icon indicating copy to clipboard operation
Chronicle-Queue copied to clipboard

Inconsistent EOF handling

Open rogersimmons opened this issue 2 years ago • 0 comments

SingleChronicleQueueStore handles writing an EOF when a cycle rolls.

The implementation has a tryReserve check, and either uses the already mapped wire (if reserve ok), or remaps a temporary wire (if reserve not ok).

The temporary wire differs in at least 2 important respects vs the already-mapped wire, which leads to inconsistent EOF behaviour. Specifically:

  • usePadding is true for the local mapping, false for the temp mapping
  • the writePosition() is off by 8 bytes between the local (higher value) and temp mapping
    public boolean writeEOF(@NotNull Wire wire, long timeoutMS) {
        throwExceptionIfClosed();

        String fileName = mappedFile.file().getAbsolutePath();

        // just in case we are about to release this
        if (wire.bytes().tryReserve(this)) {
            try {
                return writeEOFAndShrink(wire, timeoutMS);

            } finally {
                wire.bytes().release(this);
            }
        }

        try (MappedBytes bytes = MappedBytes.mappedBytes(mappedFile.file(), mappedFile.chunkSize())) {
            Wire wire0 = WireType.valueOf(wire).apply(bytes);
            return writeEOFAndShrink(wire0, timeoutMS);

        } catch (Exception e) {
            Jvm.warn().on(getClass(), "unable to write the EOF file=" + fileName, e);
            return false;
        }
    }

rogersimmons avatar May 25 '22 15:05 rogersimmons