UDP Server lost packets

[AlwaysYng]

Hello, I'm trying to use openresty as a udp server. But some packets were lost.

Centos7 nginx version: openresty/1.17.8.2

1. tcpdump: openresty received 3 packets, but only responded once

18:48:24.625532 IP 10.13.0.110.53193 > vmware.documentum: UDP, length 17
18:48:24.625568 IP 10.13.0.110.53193 > vmware.documentum: UDP, length 17
18:48:24.625571 IP 10.13.0.110.53193 > vmware.documentum: UDP, length 17
18:48:24.626090 IP vmware.documentum > 10.13.0.110.53193: UDP, length 17

2. access.log: only 1

[28/Aug/2020:18:48:24 +0800] [2020-08-28T18:48:24+08:00] 10.13.0.110 - UDP 200 17 17 0.000 "-" "-" "-" "-"

3. nginx.conf

user root;
worker_processes 1;
worker_cpu_affinity 01;
worker_rlimit_nofile 65535;
pid /run/nginx.pid;

events {
    worker_connections 65535;
    multi_accept on;
    use epoll;
}

stream {
    lua_code_cache on;

    log_format main '[$time_local] [$time_iso8601] $remote_addr - '
                     '$protocol $status $bytes_sent $bytes_received '
                     '$session_time "$upstream_addr" '
                     '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';

    access_log  /proj/log/access.log main;
    error_log   /proj/log/error.log debug;

    server {
        listen 10002 udp;
        content_by_lua_file src/udp.lua;
    }
}

4. src/udp.lua

local sock, err = ngx.req.socket()
if not sock then
    ngx.log(ngx.ERR, string.format("get socket fail. err: %s", err))
    return
end

local data, err = sock:receive()
if not data then
    ngx.log(ngx.ERR, string.format("receive fail. err: %s", err))
    return
end

ngx.sleep(0)

sock:send(data)

5. client.py

#!/bin/python

import socket
import ujson as json
import gevent
import random

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
addr = ("10.13.0.63", 10002)

for i in range(0, 3):
    sock.sendto(json.dumps({"idx": idx}), addr)
    # print(sock.recv(1024))

[dndx]

@AlwaysYng I think if you uncomment the sock.recv between packets it should work correctly.

[doujiang24]

@AlwaysYng If you insist on the pipeline mode, you can try the following way:

local sock, err = ngx.req.socket()
if not sock then
    ngx.log(ngx.ERR, string.format("get socket fail. err: %s", err))
    return
end

local max = 100
local i = 0

while true do
    local data, err = sock:receive()

    if not data then
        -- handle error ...

        -- at least "max" packages or worker exiting
        if ngx.worker.exiting() or i >= max then
            break
        end
    end

    i = i + 1

    -- handle the data
end

-- should exit from the Lua content handler
-- should not have any yield operations now, even ngx.sleep(0)

return

[doujiang24]

=== TEST 6: pipeline send
--- dgram_server_config
    content_by_lua_block {
        local sock, err = ngx.req.socket()
        if not sock then
            ngx.log(ngx.ERR, "failed to get the request socket: ", err)
            return ngx.exit(ngx.ERROR)
        end

        local all_data = ""

        while true do
            local data, err = sock:receive()
            if not data then
                ngx.log(ngx.WARN, "failed to receive: ", err)
                break
            end

            all_data = all_data .. data

            ngx.sleep(0.05)
        end

        local ok, err = sock:send("received: " .. all_data)
        if not ok then
            ngx.log(ngx.ERR, "failed to send: ", err)
            return ngx.exit(ngx.ERROR)
        end
    }

--- gen_dgram_request

    local bytes, err = sock:send('package 1')
    if not bytes then
        ngx.say("send stream request error: ", err)
        return
    end

    local bytes, err = sock:send('package 2')
    if not bytes then
        ngx.say("send stream request error: ", err)
        return
    end

--- dgram_response chomp
received: package 1package 2
--- no_error_log
[error]

I prepared a test case to reproduce this issue, I think we should fix it in the stream lua module.

[spacewander]

Isn't the behavior expected?

When the stream server is in UDP mode, reading from the downstream socket returned by the ngx.req.socket call will only return the content of a single packet. https://github.com/openresty/stream-lua-nginx-module#nginx-api-for-lua

When you call receive once, you will get only one packet.