oceanbase
oceanbase copied to clipboard
ObSQL Audit 无锁队列扩缩容
Task Description
ObMySQLRequestManager中使用 一个无锁的 ObRaQueue 记录每一个 audit 记录的指针。ObRaQueue 是无锁实现,但内存是静态申请的,导致在小租户下内存占用较大 (80M) 。目前仅在meta租户和mini mode下使用较小的queue size初始化以减少内存占用,但不能处理租户内存扩缩容场景,希望将此queue做为动态大小。
Solution Description
设计目标:
- 向上提供随机读取的接口;
- 支持内存动态扩缩容;
- 无锁实现高并发线程安全;
具体细节:
使用一个两层队列的结构,通过构造多个小的静态队列来组成一个大的动态队列。
- 节点内存:一级队列存放二级队列的指针和该二级队列上的引用计数,占用16B;二级队列存在每个Audit记录的指针,占用8B;
- 扩缩容:每次操作的粒度为一个小队列的大小(Evict Batch)。Batch设置的越小,队列的内存可控粒度越小,但malloc/free操作越频繁;
Passed Regressions
oceanbase/unittest/observer/test_ob_ra_queue.cpp oceanbase/unittest/observer/test_ob_tl_queue.cpp oceanbase/unittest/observer/test_ob_mysql_request_manager.cpp
Upgrade Compatibility
Other Information
Release Note
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
fhkong seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.