cloudberry icon indicating copy to clipboard operation
cloudberry copied to clipboard

Add udp2, an experimental IPC layer implementation

Open zhangyue1818 opened this issue 3 months ago • 6 comments

Fixes #ISSUE_Number

What does this PR do?

Type of Change

  • [ ] Bug fix (non-breaking change)
  • [ ] New feature (non-breaking change)
  • [ ] Breaking change (fix or feature with breaking changes)
  • [ ] Documentation update

Breaking Changes

Test Plan

  • [ ] Unit tests added/updated
  • [ ] Integration tests added/updated
  • [ ] Passed make installcheck
  • [ ] Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions


zhangyue1818 avatar Sep 11 '25 12:09 zhangyue1818

Awesome! Could you please share some information about the architecture of the new solution? I think there might be an original (acmsigmod?) article on how to build an effective interconnect layer. (I see some familiar elements in the code, but I cannot fully comprehend what it is)

leborchuk avatar Sep 12 '25 08:09 leborchuk

Awesome! Could you please share some information about the architecture of the new solution? I think there might be an original (acmsigmod?) article on how to build an effective interconnect layer. (I see some familiar elements in the code, but I cannot fully comprehend what it is)

During performance testing, we identified the Motion node as a significant performance bottleneck. Further analysis revealed that the root cause stems from inefficiencies in the underlying IPC layer protocol implementation. To address this, we plan to enhance the IPC layer to improve overall system performance.

Currently, the IPC layer is tightly coupled with the database kernel, which makes implementing performance optimizations challenging and limits flexibility. As an initial step, we are decoupling the IPC layer from the database kernel. This separation will not only simplify future improvements but also enable independent testing and development of the IPC layer.

This commit represents foundational work—specifically, the decoupling of the existing UDP-based IPC layer from the database kernel—and lays the groundwork for subsequent performance optimization efforts.

zhangyue1818 avatar Sep 13 '25 09:09 zhangyue1818

The following two files also need the license header:

[WARNING] Files with unapproved licenses:
  contrib/udp2/CMakeLists.txt
  contrib/udp2/ic_common/CMakeLists.txt

tuhaihe avatar Sep 16 '25 10:09 tuhaihe

7e7c028

The following two files also need the license header:

[WARNING] Files with unapproved licenses:
  contrib/udp2/CMakeLists.txt
  contrib/udp2/ic_common/CMakeLists.txt

fix in https://github.com/apache/cloudberry/commit/251adf48a6c4940ae2a221a0a870fdb4aadc2817

zhangyue1818 avatar Sep 23 '25 02:09 zhangyue1818

Could you please share some information about the architecture of the new solution by doc ?

yjhjstz avatar Oct 14 '25 20:10 yjhjstz

Could you please share some information about the architecture of the new solution by doc ?

Describes the architecture and other aspects in README.md,see 94428f1

zhangyue1818 avatar Oct 15 '25 12:10 zhangyue1818

That would be better to have a comprehensive commit body to these commits, sharing the basic information on what, why and how and more, following the .gitmessage template. @zhangyue1818

If we squash and merge this PR, then can paste the commit body manually.

tuhaihe avatar Nov 28 '25 09:11 tuhaihe