Flow running状态，sink端接收不到数据

[alexcd90]

通过dataRepartitionRdd在driver端遍历可以获取到消息数据，但是mysql的sink端收不到数据。

[alexcd90]

2019-09-23 17:59:54 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start foreachRDD 2019-09-23 17:59:59 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start doDirectiveTopic 2019-09-23 18:00:18 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start Repartition [GC (Allocation Failure) [ParNew: 31543K->2125K(33088K), 0.0077595 secs] 80794K->51636K(106444K), 0.0078571 secs] [Times: user=0.00 sys=0.02, real=0.01 secs] 2019-09-23 18:01:44 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[94] - rdd data print .......... [GC (Allocation Failure) [ParNew: 31565K->2761K(33088K), 0.0080217 secs] 81076K->52272K(106444K), 0.0081119 secs] [Times: user=0.00 sys=0.02, real=0.01 secs] [GC (Allocation Failure) [ParNew: 32200K->3648K(33088K), 0.1970078 secs] 81712K->54259K(106444K), 0.1970989 secs] [Times: user=0.12 sys=0.16, real=0.20 secs] [GC (Allocation Failure) [ParNew: 29429K->2004K(33088K), 1.2053424 secs] 80040K->70650K(106444K), 1.2054169 secs] [Times: user=1.87 sys=0.09, real=1.20 secs] [GC (CMS Initial Mark) [1 CMS-initial-mark: 68645K(73356K)] 74984K(106444K), 0.0356161 secs] [Times: user=0.02 sys=0.00, real=0.04 secs] [CMS-concurrent-mark-start] [CMS-concurrent-mark: 0.086/0.086 secs] [Times: user=0.12 sys=0.00, real=0.09 secs] [CMS-concurrent-preclean-start] [CMS-concurrent-preclean: 0.002/0.002 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] [CMS-concurrent-abortable-preclean-start] {"id": 1, "name": "test", "phone":"18074546423", "city": "Beijing", "time": "2017-12-22 10:00:00"} {"id": 1, "name": "test03", "phone":"18074542345", "city": "Beijing", "time": "2017-12-25 10:00:00"} {"id": 1, "name": "test04", "phone":"18074542346", "city": "Beijing", "time": "2017-12-27 10:00:00"} {"id": 1, "name": "test02", "phone":"18074541234", "city": "Beijing", "time": "2017-12-23 10:00:00"} {"id": 1, "name": "test05", "phone":"18174542346", "city": "Beijing", "time": "2017-12-25 10:00:00"} CMS: abort preclean due to time [CMS-concurrent-abortable-preclean: 0.353/5.011 secs] [Times: user=1.31 sys=0.80, real=5.01 secs] [GC (CMS Final Remark) [YG occupancy: 12689 K (33088 K)][Rescan (parallel) , 0.0028635 secs][weak refs processing, 0.0015901 secs][class unloading, 0.0128769 secs][scrub symbol table, 0.0142750 secs][scrub string table, 0.0005552 secs][1 CMS-remark: 68645K(73356K)] 81335K(106444K), 0.0325517 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] [CMS-concurrent-sweep-start] [CMS-concurrent-sweep: 0.029/0.029 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] [CMS-concurrent-reset-start] [CMS-concurrent-reset: 0.001/0.002 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2019-09-23 18:02:23 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start create classifyRdd [GC (Allocation Failure) [ParNew: 31444K->1136K(33088K), 0.0086701 secs] 98129K->67821K(144232K), 0.0088571 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2019-09-23 18:07:20 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start doStreamLookupData 2019-09-23 18:09:21 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start doMainData [GC (Allocation Failure) [ParNew: 30576K->1761K(33088K), 0.0057037 secs] 97261K->68446K(144232K), 0.0057792 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 2019-09-23 18:10:34 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start doOtherData 2019-09-23 18:11:18 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - start printTopicPartition 2019-09-23 18:11:19 [streaming-job-executor-0] INFO edp.wormhole.sparkx.common.SparkxUtils[54] - ----------- OffsetRange(topic: 'source', partition: 0, range: [0 -> 5]) 2019-09-23 18:11:37 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - commit offsets 2019-09-23 18:11:41 [streaming-job-executor-0] INFO edp.wormhole.sparkx.batchflow.BatchflowMainProcess[54] - write appid to zookeeper,application_1569203303506_0011

[daizikaikou]

你好，请问解决了么，我也出现了这种情况