[Enhancement] Improve routine load too many error rows message
Why I'm doing:
mysql> show routine load from kafka_test\G
*************************** 1. row ***************************
Id: 282486
Name: label0
CreateTime: 2024-05-11 15:26:20
PauseTime: 2024-05-11 15:27:15
EndTime: NULL
DbName: kafka_test
TableName: lineorder2
State: PAUSED
DataSourceType: KAFKA
CurrentTaskNum: 0
JobProperties: {"partitions":"*","rowDelimiter":"\t","partial_update":"false","columnToColumnExpr":"*","maxBatchIntervalS":"10","partial_update_mode":"null","whereExpr":"*","timezone":"Asia/Shanghai","format":"csv","columnSeparator":"'|'","log_rejected_record_num":"0","taskTimeoutSecond":"60","json_root":"","maxFilterRatio":"1.0","strict_mode":"false","js
onpaths":"","taskConsumeSecond":"15","desireTaskConcurrentNum":"5","maxErrorNum":"0","strip_outer_array":"false","currentTaskConcurrentNum":"1","maxBatchRows":"200000"}
DataSourceProperties: {"topic":"wyb_test","currentKafkaPartitions":"0","brokerList":"127.0.0.1:9093"}
CustomProperties: {"group.id":"label0_012bae3f-19f4-452f-bb32-8133ecc9f2c8"}
Statistic: {"receivedBytes":2953715,"errorRows":29657,"committedTaskNum":0,"loadedRows":0,"loadRowsRate":0,"abortedTaskNum":1,"totalRows":29657,"unselectedRows":0,"receivedBytesRate":196000,"taskExecuteTimeMs":15059} Progress: {"0":"81418"}
TimestampProgress: {"0":"1715412435492"}
ReasonOfStateChanged: ErrorReason{errCode = 102, msg='current error rows is more than max error num'}
ErrorLogUrls: http://127.0.0.1:8049/api/_load_error_log?file=error_log_6f7b01c3144a4b98_ab0e42d4b3329205
TrackingSQL: select tracking_log from information_schema.load_tracking_logs where job_id=282486
OtherMsg:
LatestSourcePosition: {"0":"62499"}
1 row in set (0.00 sec)
What I'm doing:
mysql> show routine load from kafka_test\G
*************************** 1. row ***************************
Id: 283192
Name: label0
CreateTime: 2024-05-11 16:12:36
PauseTime: 2024-05-11 16:24:56
EndTime: NULL
DbName: kafka_test
TableName: lineorder2
State: PAUSED
DataSourceType: KAFKA
CurrentTaskNum: 0
JobProperties: {"partitions":"*","rowDelimiter":"\t","partial_update":"false","columnToColumnExpr":"*","maxBatchIntervalS":"10","partial_update_mode":"null","whereExpr":"*","timezone":"Asia/Shanghai","format":"csv","columnSeparator":"'|'","log_rejected_record_num":"0","taskTimeoutSecond":"60","json_root":"","maxFilterRatio":"1.0","strict_mode":"false","jsonpaths":"","taskConsumeSecond":"15","desireTaskConcurrentNum":"5","maxErrorNum":"10000","strip_outer_array":"false","currentTaskConcurrentNum":"1","maxBatchRows":"200000"}
DataSourceProperties: {"topic":"wyb_test","currentKafkaPartitions":"0","brokerList":"127.0.0.1:9093"}
CustomProperties: {"group.id":"label0_c3e8da1a-8cc9-4df9-85c7-eb7b72e5020c"}
Statistic: {"receivedBytes":2422562,"errorRows":24345,"committedTaskNum":0,"loadedRows":0,"loadRowsRate":0,"abortedTaskNum":6,"totalRows":24345,"unselectedRows":0,"receivedBytesRate":307000,"taskExecuteTimeMs":7881}
Progress: {"0":"130452"}
TimestampProgress: {"0":"1715415895552"}
ReasonOfStateChanged: ErrorReason{errCode = 102, msg='Current error rows: 13755 is more than max error num: 10000. Check the 'TrackingSQL' field for detailed information'}
ErrorLogUrls: http://127.0.0.1:8049/api/_load_error_log?file=error_log_f3a6d9fc207f4bcd_a592dd995cb80bce
TrackingSQL: select tracking_log from information_schema.load_tracking_logs where job_id=283192
OtherMsg:
LatestSourcePosition: {"0":"124109"}
1 row in set (0.01 sec)
Fixes #issue
What type of PR is this:
- [ ] BugFix
- [ ] Feature
- [x] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [x] Yes, this PR will result in a change in behavior.
- [ ] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [x] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [ ] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [x] 3.3
- [x] 3.2
- [x] 3.1
- [ ] 3.0
- [ ] 2.5
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
[FE Incremental Coverage Report]
:white_check_mark: pass : 4 / 4 (100.00%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | com/starrocks/load/routineload/RoutineLoadJob.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/common/ErrorCode.java | 1 | 1 | 100.00% | [] |
[BE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
@Mergifyio backport branch-3.3
@Mergifyio backport branch-3.2
@Mergifyio backport branch-3.1
backport branch-3.3
✅ Backports have been created
- #45721 [Enhancement] Improve routine load too many error rows message (backport #45480) has been created for branch
branch-3.3
backport branch-3.2
✅ Backports have been created
- #45722 [Enhancement] Improve routine load too many error rows message (backport #45480) has been created for branch
branch-3.2but encountered conflicts
backport branch-3.1
✅ Backports have been created
- #45723 [Enhancement] Improve routine load too many error rows message (backport #45480) has been created for branch
branch-3.1but encountered conflicts