option to change datagram read buffer size

[arikastarvo]

Even for a tiny bit higher than nothing troughput using UDP, packets will get lost with default setting. Setting read buffer to a higher number can fix this. Tested with roughly 3k EPS (~2MB/sec) UDP syslog troughput and 4MB buffer size (worked like a charm without loss).

[AtakanColak]

@mcuadros could you review or have people with write access review this please?

[arikastarvo]

i'll leave a quick hackis test-script also:

package main

import (
    "gopkg.in/mcuadros/go-syslog.v2"
    "fmt"
    "os"
)

func main() {
    channel := make(syslog.LogPartsChannel)
    handler := syslog.NewChannelHandler(channel)

    server := syslog.NewServer()
    server.SetFormat(syslog.RFC5424)
    server.SetHandler(handler)
    //server.SetDatagramReadBufferSize(1024 * 10000)
    server.ListenUDP("127.0.0.1:5514")
    server.Boot()

    go func(channel syslog.LogPartsChannel) {
        for logParts := range channel {
            if logParts["message"] == "end" {
                os.Exit(0)
            }
            fmt.Println(logParts)
        }
    }(channel)

    server.Wait()
}

To use PR feature, just uncomment the line (currently 10mb). To test, use logger and loggen (from syslog-ng package):

# kickstart server with linecounter in one prompt
go run testserver.go | wc -l
# shoot n log entries to server
loggen --dgram --number 1000 --rate 1000 --size 512 localhost 5514
# send closing entry
logger --udp --port 5514 -n localhost --rfc5424 end

Of course OS-s UDP buffer has be big enough also. But if OS buffer is big enough, then tweaking SetDatagramReadBufferSize value will change the outcome. Small buffer size = messages get lost; bigger buffer size = messages won't get lost (within reasonable msg rate)

[AtakanColak]

@arikastarvo would be nice if this setting checked OS UDP buffer as well, if its possible.

[arikastarvo]

I don't feel quite comfortable in this area to be honest but I doubt there is a cross-OS way to do this from go. I might be wrong. For these quick tests in Ubuntu 20 VM, I just changed net.core.rmem_max kernel parameter with sysctl to a higher value.