Add channel API

[kingluo]

trafficstars

I try to add a new API about the channel (Codes are done, ready for pull request). It has the same characteristics as golang channel. You could create a new channel, with or without capacity, send messages to the channel, receive messages from the channel, close the channel, and do select upon multiple channel operations. You could share the channel between different request contexts.

Here are some examples:

Simple send and receive:

    content_by_lua_block {
        local ch = ngx.channel.new(1)
        local ok, err = ch:send("hello")
        if not ok then
            ngx.say("error")
        end
        local th = ngx.thread.spawn(function()
            local val, ok = ch:recv()
            ngx.say(ok, " : ", val)
        end)
        ngx.thread.wait(th)
    }

Select:

    content_by_lua_block {
        local ch1 = ngx.channel.new()
        local ch2 = ngx.channel.new()
        local th = ngx.thread.spawn(function()
            ngx.channel.select({
                {ch1, "recv", function(val, ok)
                    ngx.say(val, " : ", ok)
                end},
                {ch2, "send", "foobar"},
            })
        end)
        ch1:send(12)
        ngx.thread.wait(th)
    }

Select with default statement:

    content_by_lua_block {
        local ch1 = ngx.channel.new()
        local ch2 = ngx.channel.new()
        local th = ngx.thread.spawn(function()
            ngx.channel.select({
                {ch1, "recv"},
                {ch2, "send", "foobar"},
                {"default", function() ngx.say("default") end}
            })
        end)
        ch1:close()
        ngx.thread.wait(th)
    }

Are you interested in this API? Any suggestions are welcome.

[doujiang24]

Does channel work across multiple worker processes?

[kingluo]

Does channel work across multiple worker processes?

No.

[splitice]

Without using shared memory thereby supporting multiple worker processes what is the application?

[lun-4]

One usecase for adding worker processes in a channel API is providing easier integration with other Lua C libraries.

I was experimenting with run_worker_thread a while ago and to integrate with a Lua SQLite C binding, but since worker threads don't support userdata being passed (ngx.shared doesn't support it either), the experiment failed at that. The library uses an userdata instance to represent the database handle.

A channel would be suitable for this usecase (if it added worker processes), as I could just send the SQL queries I want through the channel, and receive the values through it as well, without having to block the main thread.

[kingluo]

I was experimenting with run_worker_thread a while ago and to integrate with a Lua SQLite C binding, but since worker threads don't support userdata being passed (ngx.shared doesn't support it either), the experiment failed at that. The library uses an userdata instance to represent the database handle.

I am wondering you really need to transfer userdata to worker thread? Copy query string to worker thread, and do SQLite query in worker thread, returns the result string/table to the main thread, isn't it enough yet?

Each worker thread runs lua codes in a seperate VM instance indepdent of the one in main thread, so it's sort of meaningless to share non-data types, e.g. lua function. Userdata is also special, because it is normally bound with a metatable, which contains lua/C functions. Copy a memory block is easy, but copy functions and re-establish the relationship between them and the userdata is difficult.

BTW, run_worker_thread supports ngx.shared.DICT API now.

[lun-4]

I am wondering you really need to transfer userdata to worker thread? Copy query string to worker thread, and do SQLite query in worker thread, returns the result string/table to the main thread, isn't it enough yet?

That would be a solution channels would make much easier, I don't want to create a new database connection for every SQLite query because of inefficiency (would also not let things like transactions be able to be implemented). If channels functioned with worker threads, I could have a dedicated long-lived worker thread made just for SQLite queries inside OpenResty, which would be ideal.