kop icon indicating copy to clipboard operation
kop copied to clipboard

[improve] Consolidate Netty channel flushes to mitigate syscall overhead

Open Demogorgon314 opened this issue 3 years ago • 0 comments

TODO: We still need to test to find out the best explicitFlushAfterFlushes config value.

The better option is migration the org.apache.cassandra.transport.Flusher. It can reduce the flush count and reduce the FlushConsolidationHandler latency(https://github.com/netty/netty/commit/e3c8a924993018e5d8e7d724fc89cc1f9d691387).

Motivation

When we are writing/reading a lot of small entries, one of the main bottlenecks in kop becomes CPU usage. A big part of it is caused by the syscall overhead when writing to the socket.

The reason is that we have many independent (and small) operations happening on the connection and each time we call writeAndFlush() on each of them, causing many write() calls on the socket.

Netty has a mechanism to consolidate the flushes on the channel and it improves the handling of many small entries.

Modifications

Documentation

Check the box below.

Need to update docs?

  • [ ] doc-required

    (If you need help on updating docs, create a doc issue)

  • [x] no-need-doc

    (Please explain why)

  • [ ] doc

    (If this PR contains doc changes)

Demogorgon314 avatar Jul 19 '22 03:07 Demogorgon314