Changefeeds are not distributed evenly over TiCDC nodes

[King-Dylan]

What did you do?

Creating multiple changefeeds where each one contains only one table results in all table tasks being sent to a single capture. This is unreasonable. I think before we bind a table to a capture, we should add more judgment conditions, such as considering the current tasks already bound to it. Then, we can choose the best capture to bind the table to, which will be better for this scenario. https://github.com/pingcap/tiflow/blob/master/cdc/scheduler/internal/v3/scheduler/scheduler_basic.go#L81-L82

What did you expect to see?

The tasks evenly distirbuted on different ticdc nodes.

What did you see instead?

All of them are bound on specific node.

Versions of the cluster

TiCDC version (execute cdc version):

v7.1.1

[King-Dylan]

https://github.com/pingcap/tiflow/issues/11331 seems just fixed the uneven distribute scenario for one changefeed muti-tables. This scenario is more like multiple changefeeds with one table task.

[fubinzh]

#11331 seems just fixed the uneven distribute scenario for one changefeed muti-tables. This scenario is more like multiple changefeeds with one table task.

@King-Dylan The link in the comment is wrong?

[choudharypankaj]

https://github.com/pingcap/tiflow/pull/9847 This is the previous issue that @King-Dylan was referring to.

[fubinzh]

This looks like an enhancement? Please @flowbehappy take a look.

[flowbehappy]

We are working on the new arch TiCDC https://github.com/pingcap/ticdc , will address it in the new project.