ngx.location.capture error : unsafe byte "0x9" in header

[kiss291323003]

openresty version:1.15.8.3 example code:

access_by_lua_block {
local res = ngx.location.capture('/secureCheck',
                    {
                        method = method,
                        body = args.data,
                        copy_all_vars = true,
                        always_forward_body = true,
                        share_all_vars = true
                    }
                )
}

full error:

access_by_lua(nginx.conf:223):15: in main chunk, client: 100.121.135.117, server: , request: "GET xxxxxxxxxx HTTP/1.1", host: "******", referrer: "*****"
2020/05/28 10:49:13 [error] 31820#0: *6897729 unsafe byte "0x9" in header "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) \x09\x09\x09Chrome/55.0.2883.95 Safari/537.36", client: 100.121.135.66, server: , request: "GET xxxxx HTTP/1.1", host: "xxxxx", referrer: "blank"
2020/05/28 10:49:13 [error] 31820#0: *6897729 lua entry thread aborted: runtime error: access_by_lua(nginx.conf:223):15: failed to adjust the subrequest: -1
stack traceback:
coroutine 0:
 [C]: in function 'capture'
 access_by_lua(nginx.conf:223):15: in main chunk, client: 100.121.135.66, server: , request: "GET xxxxxxxx HTTP/1.1", host: "xxxxxxx", referrer: "blank"

[doujiang24]

@kiss291323003 can you provide a test request to reproduce it?

[kiss291323003]

sorry i cant provide any test request , but it do realy occur in my produce error log

[kiss291323003]

reproduce conf

worker_processes  32;
events
{
    worker_connections 65535;
}
http
{
    init_by_lua_block {
        httpMethodSwitch =
        {
            ['GET'] = function()
                return ngx.HTTP_GET
            end,
            ['HEAD'] = function()
                return ngx.HTTP_HEAD
            end,
            ['PUT'] = function()
                return ngx.HTTP_PUT
             end,
            ['POST'] = function()
                return ngx.HTTP_POST
            end,
            ['DELETE'] = function()
                return ngx.HTTP_DELETE
            end,
            ['OPTIONS'] = function()
                return ngx.HTTP_OPTIONS
            end,
            ['MKCOL'] = function()
                return ngx.HTTP_MKCOL
            end,
            ['COPY'] = function()
                return ngx.HTTP_COPY
            end,
            ['MOVE'] = function()
                return ngx.HTTP_MOVE
            end,
            ['PROPFIND'] = function()
                return ngx.HTTP_PROPFIND
            end,
            ['PROPPATCH'] = function()
                return ngx.HTTP_PROPPATCH
            end,
            ['LOCK'] = function()
                return ngx.HTTP_LOCK
            end,
            ['UNLOCK'] = function()
                return ngx.HTTP_UNLOCK
            end,
            ['PATCH'] = function()
                return ngx.HTTP_PATCH
            end,
            ['TRACE'] = function()
                return ngx.HTTP_TRACE
            end
        };

        function pushStatistics(premature,status,requestMethod,responseLength)

        end
    }
    # 默认上游网关
    upstream defaultUpStream
    {
        server 127.0.0.1:9501 weight=10 max_fails=3 fail_timeout=360s ;
        #server 172.16.0.22:80 weight=10 max_fails=3 fail_timeout=360s ;
        #server 172.16.0.25:80 weight=10 max_fails=3 fail_timeout=360s ;
        #server 172.16.0.26:80 weight=10 max_fails=3 fail_timeout=360s ;
        #server 172.16.0.27:80 weight=10 max_fails=3 fail_timeout=360s ;
        #server 172.16.0.28:80 weight=8  max_fails=3 fail_timeout=360s ;
        #server 172.16.0.29:80 weight=10 max_fails=3 fail_timeout=360s ;
        #server 172.16.0.30:80 weight=10 max_fails=3 fail_timeout=360s ;
    }

    server
    {
        set $wafHost  http://127.0.0.1:9501;
        client_max_body_size 20m;
        resolver 223.5.5.5;
        location ~ .*\.(gif|jpg|jpeg|png|bmp|svg|swf|js|css|mp3|mp4)$
        {
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-HOST $host;
            proxy_pass http://defaultUpStream;
            break;
        }

        location /
        {
            set $requestUri '';
            access_by_lua_block
            {
                -- 强制隐藏服务端Server信息
                ngx.header['Server'] = "Fosuss WAF";
                ngx.header['X-Server'] = "Fosuss WAF";
                ngx.var.requestUri = ngx.var.request_uri;
                ngx.req.read_body();
                if (ngx.req.get_method() ~= "GET") and (ngx.req.get_method() ~= "POST") then
                    return;
                end
                -- 请求方法判断
                local method = "POST";
                local f = httpMethodSwitch[ngx.req.get_method()]
                if(f) then
                     method = f();
                end
                local args, err = ngx.req.get_uri_args();
                -- 安全检查
                local res = ngx.location.capture('/secureCheck',
                    {
                        method = method,
                        body = args.data,
                        copy_all_vars = true,
                        always_forward_body = true,
                        share_all_vars = true
                    }
                );
                -- waf 挂了,return ，直接到真实后端。
                if res.status == 502 then
                    return;
                end
                -- waf返回非200
                if not (res.status == ngx.HTTP_OK) then
                    ngx.status = res.status;
                    for k,v in pairs(res.header) do
                        ngx.header[k] = v;
                    end
                    ngx.say(res.body);
                    ngx.exit(res.status)
                end
                -- cookie设置
                if ngx.header['Set-Cookie'] then
                     ngx.header['Set-Cookie'] = res.header['Set-Cookie'];
                end
                -- upstream 设置
                if res.header['Up-Stream'] then
                    ngx.var.finalUpStream = res.header['Up-Stream'];
                end
            }
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Real-HOST $host;
            proxy_pass http://defaultUpStream;
            proxy_pass_request_body on;
            proxy_pass_request_headers on;
            proxy_connect_timeout       360;
            proxy_send_timeout          360;
            proxy_read_timeout          360;
            send_timeout                360;
            log_by_lua_block
            {
                ngx.timer.at(0, pushStatistics,ngx.var.status, ngx.var.request_method, ngx.var.bytes_sent)
            }
        }

        location /secureCheck
        {
            internal;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Real-HOST $host;
            proxy_pass $wafHost$requestUri;
            proxy_pass_request_body on;
            proxy_pass_request_headers on;
            proxy_connect_timeout       360;
            proxy_send_timeout          360;
            proxy_read_timeout          360;
            send_timeout                360;
        }

        error_log error.log;
        access_log off;
    }
}

reproduce client using php code

$ua = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) \x09\x09\x09Chrome/55.0.2883.95 Safari/537.36";

$headerArray =array("User-Agent:".$ua);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://127.0.0.1");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch,CURLOPT_HTTPHEADER,$headerArray);
$output = curl_exec($ch);
curl_close($ch);

[kiss291323003]

temporary solution：

                -- 特殊UA检查
                if ngx.var.http_user_agent then
                    local specialHeader = string.find(ngx.var.http_user_agent, "\x09");
                    if specialHeader then
                       return
                    end
                end

[kiss291323003]

is that any one hears me ???

[agentzh]

@kiss291323003 The character \x09 is a tab character (which is often used as source code indentation, for example). We'll whitelist it in the next release of OpenResty which will be 1.17.8.1. Thanks for the report. It's weird that your clients' user agent header value uses tab characters BTW.

[kollol-chowdhury-incontact]

Hi,

Was this whitelisted in OpenResty latest version. I am using 1.19.9.1 and still I am getting this error.

Thanks

[zhuizhuhaomeng]

why does your header contain 0x09? it is invalid.

[zhuizhuhaomeng]

have a look at this URL https://github.com/openresty/lua-nginx-module#ngxreqset_header

[kiss291323003]

I use openresty to implement an waf 。and the hack will use some special character to attack my web server , and some of the CVE also using special character . and I hope when this special character occur, I can handler by my lua code but no openresty error