doris-spark-connector
doris-spark-connector copied to clipboard
Add a parameter that controls the number of StreamLoad tasks committed per partition #92
Add a parameter that controls the number of StreamLoad tasks committed per partition
Issue Number: close #92
- Does it affect the original behavior: (I Don't know)
- Has unit tests been added: (No)
- Has document been added or modified: (Yes)
- Does it need to update dependencies: (Yes)
- Are there any changes that cannot be rolled back: (No)
Thank you for your contribution, can you resolve the conflict?
If the data is processed according to the partition, when a single partition is particularly large, there may be problems in a streamload?
If the data is processed according to the partition, when a single partition is particularly large, there may be problems in a streamload?
Yes, for this problem, we need to prompt the user in the parameter description that "reparation needs to be used to ensure that the data volume per partition is in a reasonable range after this parameter is enabled". Do you think it is appropriate?