alibaba
Event driven networking interfaces for module system
See The original pull request of vanilla version for the original comments, I'm copying them as well.
Export event loop and certain networking related interfaces to module. A demonstration module 'hellonet.c' is added to demonstrate how to use these new set of module interfaces to write event driven commands without resorting to extra thread.
More comment on this:
The reason behind all the new interfaces being added comes from another project I'm doing which use something similar to php-fpm to write module business logic outside of redis-server process.
Being able to run logic externally adds these benefits:
Serious bug will crash the worker process only instead of the whole redis-server which typically have lots of state and normally take some time to reload. Business logic can be implemented in any language using the common protocol. Python for example if we care about the rapid development and runtime performance is not quite an issue. If we care more about performance and parallelism and concurrency, we can write it in golang which is simpler and usually good enough for writing business logic. The worker process can have directly connected clients attached to redis-server and use the same API developers are familiar with to talk to redis-server. With these new interface, some more interesting modules can be developed. For example, we can use nonblocking MySQL c api to connect redis-server with an mysql server, use redis as L1 cache and if cache misses reload the data from mysql server. This can simplify the typical web development task where people normally need to write cache management code to check redis for cached data and if it's missed, reload data from mysql and cache the response in redis. Now all these can be implemented in nonblocking way at a single place with the high performance event loop that already exists inside redis-server.
Thanks for labeling this pull request, I will put more design and proposal later on this thread. And here is something I think worth mentioning. Couple years ago, I wrote a social graph database architectured after Facebook's TAO, but I used redis instead of memcached. At that time there was no module system, so I had to change redis core code which is not good. After module system came out, I think it would be a much better idea to implement all these in a separate module. Since talking to mysql and upstream graph database node involves in quite bit of networking, I figure it would be great to be able to use the high performance event loop that already exists inside redis server. So I added these new interfaces.
Proposed Features:
Event Loop:
Export redis server event loop to module and execute a callback function when a specific event occurs on a file descriptor or a scheduled timeout has been reached.
RedisModule_CreateFileEvent
RedisModule_DeleteFileEvent
RedisModule_CreateTimeEvent
RedisModule_DeleteTimeEvent
Networking:
Export socket related utility functions to module to make writing portable socket code easier.
Client:
Nonblocking connect which is required to not block redis server when connection has to be made in the middle of dispatching a command. Both Tcp and Unix addresses are supported.
RedisModule_TcpNonBlockingConnect
RedisModule_UnixNonBlockingConnect
Server:
Support writing Server using TCP over IPv4 and IPv6 or Unix domain socket
RedisModule_TcpServer
RedisModule_Tcp6Server
RedisModule_UnixServer
RedisModule_TcpAccept
RedisModule_UnixAccept
Socket options:
Misc. important options related to socket programing.
RedisModule_EnableNonBlock
RedisModule_DisableNonBlock
RedisModule_EnableTcpNoDelay
RedisModule_DisableTcpNoDelay
RedisModule_TcpKeepAlive
Misc. socket functions:
RedisModule_PeerName: get address of connected peer
RedisModule_SockName: get socket name
Utility:
Context:
Module can attach some context data which can be stored and accessed with key anywhere a RedisModuleCtx is available. An optional destructor callback function can be setup at each attachment basis and will be called with RedisModuleCtx and the attachment itself as arguments when the attachment is detached explicitly or implicitly by ether new RedisModule_Attach call with the same key or unloading module without detaching first. In addition, there is a special attachment called the default one when the key of the attachment is NULL and length of key is 0, it can be accessed directly without hash lookup compared to normal attachment.
RedisModule_Attach
RedisModule_Detach
RedisModule_GetAttachment
Redis Client:
Directly connected client to redis server. The returned client file descriptor can be used to interact with redis server.
RedisModule_CreateClient
RedisModule_FreeClient
Use case scenarios:
Module with IO:
Sometime, there is need to initiate IO requests to external system. Since IO request usually takes nondeterministic time to finish or timeout eventually, either extra thread or IO multiplexing is required to execute the request asynchronously without blocking the command dispatching thread, the redis sever thread in this case. Since multithread incurs more cost than IO multiplexing, using event driven approach is important for performance critical applications. With Event Loop and Networking Interfaces, people can implement such application efficiently using high performance redis server event loop with minimum overhead. Some widely used software have nonblocking API which can be hooked into event loop directly, for example MariaDB's Non-Blocking MySQL Client Library and hiredis async client. Even if no such client is available, a hand written client using socket API directly can be implemented if the protocol is clear and easy to write.
Embed redis as caching layer:
For server applications that need caching large amount of data, writing a fast, reliable and feature rich caching layer isn't trivial. Redis satisfies all the requirements and can be a perfect caching layer of such applications, however deploying a separate redis server adds complexity of deployment. In addition, application need to worry about cases such as reconnecting if connection is broken or handling doubtable failure when connection is broken in the middle of executing commands. Even if all such cases are handled carefully, network latency is inevitable due to physical limitations. Embedding redis directly can get around many issues on the other hand. I tried to patch and rebuild redis-server itself as an shared library to do such embedding before module system came out. Now, With module system, I can move all my server's logic into a redis module, and let redis-server host my code which is reasonable and works perfectly. With the server interface exported, I can even let my module start a server and let the same original client to use the original protocol to talk to my server, yet still being able to talk to the hosting redis server easily. If the server starts other threads, using the direct connected client's file descriptor to talk to hosting redis server can make multi threaded programming a lot easier, and don't even need to bother locking and if a module function is not thread safe and must be called on behalf of redis server event thread.
Published a module Redis Process Manager (rpm) that uses these new interfaces to write module in external process. A golang framework for writing such module process is included.
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.
