bireme
bireme copied to clipboard
HashDataInc
当同步的表个数过多时,数据不能同步.
这些日志完全看不懂
[admin@bireme ~]$ tail -f app/bireme/logs/bireme.err
18:04:56,062 DEBUG QueuedThreadPool - queue acceptor-0@73163d48
18:04:56,063 INFO AbstractConnector - Started ServerConnector@1e6a3214{HTTP/1.1,[http/1.1]}{10.62.2.8:9090}
18:04:56,063 DEBUG AbstractLifeCycle - STARTED @9794ms ServerConnector@1e6a3214{HTTP/1.1,[http/1.1]}{10.62.2.8:9090}
18:04:56,063 INFO Server - Started @9794ms
18:04:56,063 DEBUG AbstractLifeCycle - STARTED @9794ms org.eclipse.jetty.server.Server@49049a04
18:04:56,064 DEBUG QueuedThreadPool - run acceptor-0@73163d48
18:07:47,905 TRACE ConsumerCoordinator - Sending OffsetCommit request with {maxwell_topic-0=OffsetAndMetadata{offset=28341, metadata=''}} to coordinator localhost:9091 (id: 2147483647 rack: null) for group bireme
18:07:47,906 TRACE NetworkClient - Sending OFFSET_COMMIT {group_id=bireme,group_generation_id=-1,member_id=,retention_time=-1,topics=[{topic=maxwell_topic,partitions=[{partition=0,offset=28341,metadata=}]}]} to node 2147483647.
18:07:47,908 TRACE NetworkClient - Completed receive from node 2147483647, for key 8, received {throttle_time_ms=0,responses=[{topic=maxwell_topic,partition_responses=[{partition=0,error_code=0}]}]}
18:07:47,909 DEBUG ConsumerCoordinator - Group bireme committed offset 28341 for partition maxwell_topic-0
[admin@bireme ~]$ tail -f app/bireme/logs/bireme.out
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_report.report_record_attr.
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_campaign.campaign_instance_attr.
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_cs.service_order_category.
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_auth.privilege_resource.
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_cs.device_sales_registration.
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_layout.share_info.
18:04:54 INFO TaskGenerator - TaskGenerator Start.
18:04:54 INFO ChangeLoader - Loader Start, corresponding table vm_campaign.redpack_record.
18:04:54 INFO Dispatcher - Dispatcher Start.
18:04:54 INFO Provider maxwell1 - Provider maxwell1 Start.
===============================================
[admin@bireme ~]$ tail -f app/bireme/logs/bireme.err
18:14:17,050 INFO ContextHandler - Stopped o.e.j.s.h.ContextHandler@3401a114{/vm_store.mijia_order_sku,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED o.e.j.s.h.ContextHandler@3401a114{/vm_store.mijia_order_sku,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping o.e.j.s.h.ContextHandler@655ef322{/vm_campaign.campaign_present,null,SHUTDOWN}
18:14:17,050 DEBUG AbstractHandler - stopping o.e.j.s.h.ContextHandler@655ef322{/vm_campaign.campaign_present,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping cn.hashdata.bireme.StateServer$StateHandler@7e276594
18:14:17,050 DEBUG AbstractHandler - stopping cn.hashdata.bireme.StateServer$StateHandler@7e276594
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED cn.hashdata.bireme.StateServer$StateHandler@7e276594
18:14:17,050 INFO ContextHandler - Stopped o.e.j.s.h.ContextHandler@655ef322{/vm_campaign.campaign_present,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED o.e.j.s.h.ContextHandler@655ef322{/vm_campaign.campaign_present,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping o.e.j.s.h.ContextHandler@73173f63{/vm_channel.channel_delivery_info,null,SHUTDOWN}
18:14:17,050 DEBUG AbstractHandler - stopping o.e.j.s.h.ContextHandler@73173f63{/vm_channel.channel_delivery_info,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping cn.hashdata.bireme.StateServer$StateHandler@55562aa9
18:14:17,050 DEBUG AbstractHandler - stopping cn.hashdata.bireme.StateServer$StateHandler@55562aa9
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED cn.hashdata.bireme.StateServer$StateHandler@55562aa9
18:14:17,050 INFO ContextHandler - Stopped o.e.j.s.h.ContextHandler@73173f63{/vm_channel.channel_delivery_info,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED o.e.j.s.h.ContextHandler@73173f63{/vm_channel.channel_delivery_info,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping o.e.j.s.h.ContextHandler@25e2ab5a{/vm_wares.unilife_prod_info_attr,null,SHUTDOWN}
18:14:17,050 DEBUG AbstractHandler - stopping o.e.j.s.h.ContextHandler@25e2ab5a{/vm_wares.unilife_prod_info_attr,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping cn.hashdata.bireme.StateServer$StateHandler@35e5d0e5
18:14:17,050 DEBUG AbstractHandler - stopping cn.hashdata.bireme.StateServer$StateHandler@35e5d0e5
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED cn.hashdata.bireme.StateServer$StateHandler@35e5d0e5
18:14:17,050 INFO ContextHandler - Stopped o.e.j.s.h.ContextHandler@25e2ab5a{/vm_wares.unilife_prod_info_attr,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED o.e.j.s.h.ContextHandler@25e2ab5a{/vm_wares.unilife_prod_info_attr,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping o.e.j.s.h.ContextHandler@3d9f6567{/vm_channel.channel_purchase_after_service_info,null,SHUTDOWN}
18:14:17,050 DEBUG AbstractHandler - stopping o.e.j.s.h.ContextHandler@3d9f6567{/vm_channel.channel_purchase_after_service_info,null,UNAVAILABLE}
18:14:17,050 DEBUG AbstractLifeCycle - stopping cn.hashdata.bireme.StateServer$StateHandler@c055c54
18:14:17,050 DEBUG AbstractHandler - stopping cn.hashdata.bireme.StateServer$StateHandler@c055c54
18:14:17,050 DEBUG AbstractLifeCycle - STOPPED cn.hashdata.bireme.StateServer$StateHandler@c055c54
```
```
[admin@bireme ~]$ tail -f app/bireme/logs/bireme.err
......................................
...............................
lue=199 bytes)), (record=DefaultRecord(offset=82670, timestamp=1520391951184, key=72 bytes, value=169 bytes)), (record=DefaultRecord(offset=82671, timestamp=1520391951184, key=72 bytes, value=186 bytes)), (record=DefaultRecord(offset=82672, timestamp=1520391951184, key=72 bytes, value=199 bytes)), (record=DefaultRecord(offset=82673, timestamp=1520391951184, key=72 bytes, value=186 bytes)), (record=DefaultRecord(offset=82674, timestamp=1520391951184, key=72 bytes, value=169 bytes)), (record=DefaultRecord(offset=82675, timestamp=1520391951184, key=72 bytes, value=169 bytes))]}]}]}
18:14:28,424 DEBUG Fetcher - Fetch READ_UNCOMMITTED at offset 78640 for partition maxwell_topic-0 returned fetch data (error=NONE, highWaterMark=572082, lastStableOffset = -1, logStartOffset = 0, abortedTransactions = null, recordsSizeInBytes=1048576)
18:14:28,424 TRACE Fetcher - Preparing to read 1048576 bytes of data for partition maxwell_topic-0 with offset 78640
18:14:28,424 TRACE Fetcher - Updating high watermark for partition maxwell_topic-0 to 572082
18:14:28,425 TRACE Fetcher - Returning fetched records at offset 78640 for assigned partition maxwell_topic-0 and update position to 79140
18:14:28,425 TRACE Fetcher - Returning fetched records at offset 79140 for assigned partition maxwell_topic-0 and update position to 79640
18:14:28,426 TRACE Fetcher - Returning fetched records at offset 79640 for assigned partition maxwell_topic-0 and update position to 80140
```
```
[admin@bireme ~]$ tail -f app/bireme/logs/bireme.out
18:15:46 INFO ChangeLoader - Chang to passimistic mode.
18:16:16 INFO ChangeLoader - Chang to passimistic mode.
18:16:31 INFO ChangeLoader - Chang to passimistic mode.
18:16:37 INFO ChangeLoader - Chang to passimistic mode.
18:16:46 INFO ChangeLoader - Chang to passimistic mode.
18:16:53 INFO ChangeLoader - Chang to passimistic mode.
18:16:55 INFO ChangeLoader - Chang to passimistic mode.
18:17:20 INFO ChangeLoader - Chang to passimistic mode.
18:17:51 INFO ChangeLoader - Chang to passimistic mode.
18:17:52 INFO ChangeLoader - Chang to passimistic mode.
18:19:23 INFO ChangeLoader - Chang to optimistic mode.
```
Several bugs has been identified in v1.0. Please use latest code from master branch. The master branch is currently under stress test and will release soon.
It is still under stress test and will not release until stable. We currently do not have enough hands and any contribution will be welcomed.
@wangzw master代码也还是会出现同样不能同步的问题. 运行一段时间后控制台消息就不动了, 像睡眠了一样. 但比1.0版本还是有所改进.
另外咨询下: 1.同一个表的数据在插入的时候怎么保证先后顺序? 如果插入失败的话, 后面的数据处理机制是怎样的? 2.触发目标库DML操作的阀值条件是什么?