A机器一个jraft服务(192.168.5.98:8086), B机器两个jraft服务(192.168.5.98:8084,192.168.5.98:8085)
- 两台机器所有服务先关闭
- 集群连接成功的启动顺序:首先启动A机器,然后再启动B机器。或者一起启动AB机器。
- 启动失败的启动顺序:先启动B机器,B机器内部有两个jraft,集群没有连接成功,出现概率问题,需要等分别几秒,十几秒,二十秒,或者两分钟,就会自动连接成功。如果马上启动A机器,就会马上连接成功。
同时启动两台jraft服务代码, new 两个实例
1、这个是启动B机器,同时启动两个jraft, 启动报错,集群没有连接成功
Hibernate: SELECT PARTITION_NAME 'name',PARTITION_DESCRIPTION 'describe' FROM information_schema.PARTITIONS WHERE table_schema=schema() AND table_name=? AND PARTITION_NAME <> 'pmax' ORDER BY PARTITION_DESCRIPTION ASC
[2022-05-19 20:41:56,215][INFO][main][com.alipay.sofa.jraft.util.JRaftServiceLoader]: SPI service [com.alipay.sofa.jraft.JRaftServiceFactory - com.alipay.sofa.jraft.core.DefaultJRaftServiceFactory] loading.
Hibernate: SELECT PARTITION_NAME 'name',PARTITION_DESCRIPTION 'describe' FROM information_schema.PARTITIONS WHERE table_schema=schema() AND table_name=? AND PARTITION_NAME <> 'pmax' ORDER BY PARTITION_DESCRIPTION ASC
[2022-05-19 20:41:56,253][INFO][main][com.alipay.sofa.jraft.util.JRaftServiceLoader]: SPI service [com.alipay.sofa.jraft.rpc.RaftRpcFactory - com.alipay.sofa.jraft.rpc.impl.BoltRaftRpcFactory] loading.
Sofa-Middleware-Log SLF4J : Actual binding is of type [ com.alipay.remoting Log4j2 ]
[2022-05-19 20:41:56,316][INFO][main][com.alipay.sofa.common.log.ReportUtil]: Sofa-Middleware-Log SLF4J : Actual binding is of type [ com.alipay.remoting Log4j2 ]
[2022-05-19 20:41:56,461][INFO][main][com.alipay.sofa.jraft.util.JRaftServiceLoader]: SPI service [com.alipay.sofa.jraft.util.timer.RaftTimerFactory - com.alipay.sofa.jraft.util.timer.DefaultRaftTimerFactory] loading.
[2022-05-19 20:41:56,465][INFO][main][com.alipay.sofa.jraft.core.NodeImpl]: The number of active nodes increment to 1.
[2022-05-19 20:41:56,621][INFO][JRaft-Closure-Executor-0][com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage]: Truncated prefix logs in data path: /database/jraft/master/data\log from log index 0 to 110, cost 1 ms.
[2022-05-19 20:41:56,626][INFO][main][com.alipay.sofa.jraft.core.FSMCallerImpl]: Starts FSMCaller successfully.
[2022-05-19 20:41:56,636][INFO][main][com.alipay.sofa.jraft.storage.snapshot.SnapshotExecutorImpl]: Loading snapshot, meta=last_included_index: 109
last_included_term: 389
peers: "192.168.5.98:8084"
peers: "192.168.5.98:8085"
peers: "192.168.5.98:8086"
.
[2022-05-19 20:41:56,655][INFO][JRaft-FSMCaller-Disruptor-0][com.alipay.sofa.jraft.core.StateMachineAdapter]: onConfigurationCommitted: 192.168.5.98:8084,192.168.5.98:8085,192.168.5.98:8086.
[2022-05-19 20:41:56,658][INFO][JRaft-FSMCaller-Disruptor-0][com.alipay.sofa.jraft.storage.snapshot.SnapshotExecutorImpl]: Node <TEST2/192.168.5.98:8084> onSnapshotLoadDone, last_included_index: 109
last_included_term: 389
peers: "192.168.5.98:8084"
peers: "192.168.5.98:8085"
peers: "192.168.5.98:8086"
[2022-05-19 20:41:56,658][INFO][main][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> target priority value has changed from: 0, to: -1.
[2022-05-19 20:41:56,659][INFO][JRaft-Closure-Executor-1][com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage]: Truncated prefix logs in data path: /database/jraft/master/data\log from log index 110 to 110, cost 0 ms.
[2022-05-19 20:41:56,716][INFO][main][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> init, term=389, lastLogId=LogId [index=109, term=389], conf=192.168.5.98:8084,192.168.5.98:8085,192.168.5.98:8086, oldConf=.
[2022-05-19 20:41:56,968][INFO][main][com.alipay.sofa.jraft.RaftGroupService]: Start the RaftGroupService successfully.
[2022-05-19 20:41:56,968][INFO][main][com.visec.platform.webstarter.cluster.core.jraft.server.RaftServer]: Started counter server at port:8084
[2022-05-19 20:41:56,969][INFO][main][com.alipay.sofa.jraft.core.NodeImpl]: The number of active nodes increment to 2.
[2022-05-19 20:41:57,028][INFO][JRaft-Closure-Executor-2][com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage]: Truncated prefix logs in data path: /database/jraft/backup/data\log from log index 0 to 110, cost 1 ms.
[2022-05-19 20:41:57,030][INFO][main][com.alipay.sofa.jraft.core.FSMCallerImpl]: Starts FSMCaller successfully.
[2022-05-19 20:41:57,031][INFO][main][com.alipay.sofa.jraft.storage.snapshot.SnapshotExecutorImpl]: Loading snapshot, meta=last_included_index: 109
last_included_term: 389
peers: "192.168.5.98:8084"
peers: "192.168.5.98:8085"
peers: "192.168.5.98:8086"
.
[2022-05-19 20:41:57,033][INFO][JRaft-FSMCaller-Disruptor-0][com.alipay.sofa.jraft.core.StateMachineAdapter]: onConfigurationCommitted: 192.168.5.98:8084,192.168.5.98:8085,192.168.5.98:8086.
[2022-05-19 20:41:57,034][INFO][JRaft-FSMCaller-Disruptor-0][com.alipay.sofa.jraft.storage.snapshot.SnapshotExecutorImpl]: Node <TEST2/192.168.5.98:8085> onSnapshotLoadDone, last_included_index: 109
last_included_term: 389
peers: "192.168.5.98:8084"
peers: "192.168.5.98:8085"
peers: "192.168.5.98:8086"
[2022-05-19 20:41:57,034][INFO][main][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> target priority value has changed from: 0, to: -1.
[2022-05-19 20:41:57,035][INFO][JRaft-Closure-Executor-3][com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage]: Truncated prefix logs in data path: /database/jraft/backup/data\log from log index 110 to 110, cost 0 ms.
[2022-05-19 20:41:57,035][INFO][main][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> init, term=389, lastLogId=LogId [index=109, term=389], conf=192.168.5.98:8084,192.168.5.98:8085,192.168.5.98:8086, oldConf=.
[2022-05-19 20:41:57,037][INFO][main][com.alipay.sofa.jraft.RaftGroupService]: Start the RaftGroupService successfully.
[2022-05-19 20:41:57,038][INFO][main][com.visec.platform.webstarter.cluster.core.jraft.server.RaftServer]: Started counter server at port:8085
Hibernate: update sys_raft_cluster set c_balance_mode=?, c_cluster_mode=?, c_cluster_status=?, c_endpoints=?, c_endpoints_vip=?, c_group_id=?, c_ip=?, c_joined=?, c_license_port=?, c_master=?, c_node=?, c_node2=?, c_other_endpoints_vip=?, c_port=?, c_status=?, c_sync_status=?, c_vip_iface=? where c_id=?
Hibernate: select sysraftclu0_.c_id as c_id1_10_0_, sysraftclu0_.c_balance_mode as c_balanc2_10_0_, sysraftclu0_.c_cluster_mode as c_cluste3_10_0_, sysraftclu0_.c_cluster_status as c_cluste4_10_0_, sysraftclu0_.c_endpoints as c_endpoi5_10_0_, sysraftclu0_.c_endpoints_vip as c_endpoi6_10_0_, sysraftclu0_.c_group_id as c_group_7_10_0_, sysraftclu0_.c_ip as c_ip8_10_0_, sysraftclu0_.c_joined as c_joined9_10_0_, sysraftclu0_.c_license_port as c_licen10_10_0_, sysraftclu0_.c_master as c_maste11_10_0_, sysraftclu0_.c_node as c_node12_10_0_, sysraftclu0_.c_node2 as c_node13_10_0_, sysraftclu0_.c_other_endpoints_vip as c_other14_10_0_, sysraftclu0_.c_port as c_port15_10_0_, sysraftclu0_.c_status as c_statu16_10_0_, sysraftclu0_.c_sync_status as c_sync_17_10_0_, sysraftclu0_.c_vip_iface as c_vip_i18_10_0_ from sys_raft_cluster sysraftclu0_ where sysraftclu0_.c_id=?
Hibernate: select sysraftmsg0_.c_key as c_key1_11_, sysraftmsg0_.c_cluster_mode as c_cluste2_11_, sysraftmsg0_.c_msg_time as c_msg_ti3_11_, sysraftmsg0_.c_node as c_node4_11_, sysraftmsg0_.c_status as c_status5_11_, sysraftmsg0_.c_update_time as c_update6_11_ from sys_raft_cluster_msg sysraftmsg0_ where sysraftmsg0_.c_status=?
Hibernate: select syssecurit0_.c_serial_number as c_serial1_18_, syssecurit0_.c_alias as c_alias2_18_, syssecurit0_.c_create_time as c_create3_18_, syssecurit0_.c_p12 as c_p4_18_, syssecurit0_.c_remark as c_remark5_18_, syssecurit0_.c_validity as c_validi6_18_ from sys_security_user_cert syssecurit0_
Hibernate: select syssecurit0_.c_code as c_code1_15_, syssecurit0_.c_desc as c_desc2_15_, syssecurit0_.c_title as c_title3_15_ from sys_security_pre_task syssecurit0_ where syssecurit0_.c_code in (? , ? , ?)
Hibernate: select permission0_.c_pre_task_id as c_pre_ta5_14_1_, permission0_.c_id as c_id1_14_1_, permission0_.c_id as c_id1_14_0_, permission0_.c_permission_id as c_permis2_14_0_ from sys_security_permission permission0_ where permission0_.c_pre_task_id in (select syssecurit0_.c_code from sys_security_pre_task syssecurit0_ where syssecurit0_.c_code in (? , ? , ?))
[2022-05-19 20:41:57,516][INFO][main][com.visec.platform.webstarter.log.core.comm.module.LogAlarmModule]: 告警日志模块状态:false
[2022-05-19 20:41:57,517][INFO][main][com.visec.platform.webstarter.log.core.comm.module.LogManagerModule]: 操作日志初始化状态:true
[2022-05-19 20:41:57,517][INFO][main][com.visec.platform.webstarter.log.LogModule]: 日志模块初始化完成
Hibernate: insert into sys_security_user (c_create_time, c_department, c_email, c_identity_id, c_last_login_time, c_lock, c_login_errors, c_nickname, c_password, c_user_pre_task_id, c_pwd_change_time, c_user_cert_id, c_username) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[2022-05-19 20:41:58,071][WARN][main][org.hibernate.engine.jdbc.spi.SqlExceptionHelper]: SQL Error: 1062, SQLState: 23000
[2022-05-19 20:41:58,072][ERROR][main][org.hibernate.engine.jdbc.spi.SqlExceptionHelper]: Duplicate entry 'admin' for key 'sys_security_user.UK_610qqx8bogkimgjuk0cesbl2n'
[2022-05-19 20:41:58,104][ERROR][main][com.visec.platform.web.LifeCycleRunner]: 开启模块失败:业务-采集模块 状态:START msg:{"state":false,"data":javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement}
[2022-05-19 20:41:58,104][INFO][main][com.visec.platform.web.LifeCycleRunner]: 关闭业务模块:[]
[2022-05-19 20:41:58,104][INFO][main][com.visec.platform.web.LifeCycleRunner]: 切换html资源映射:old:ResourceHttpRequestHandler [URL [file:G:/project/example/html/]] now:file:G:\project\example\html\
Hibernate: SELECT count() FROM information_schema.TABLES WHERE table_schema=schema() AND table_name=? AND ENGINE='InnoDB'
Hibernate: SELECT count() FROM information_schema.TABLES WHERE table_schema=schema() AND table_name=? AND ENGINE='InnoDB'
Hibernate: SELECT count() FROM information_schema.PARTITIONS WHERE table_schema=schema() AND table_name=? AND !isnull(PARTITION_METHOD)
Hibernate: SELECT count() FROM information_schema.PARTITIONS WHERE table_schema=schema() AND table_name=? AND !isnull(PARTITION_METHOD)
[2022-05-19 20:41:58,390][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 389 start preVote.
[2022-05-19 20:41:58,409][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 389 start preVote.
[2022-05-19 20:41:59,410][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:41:59,411][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:41:59,412][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:41:59,412][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:41:59,417][INFO][Bolt-conn-event-executor-13-thread-1][com.alipay.sofa.jraft.rpc.impl.core.ClientServiceConnectionEventProcessor]: Peer 192.168.5.98:8084 is connected
[2022-05-19 20:41:59,417][INFO][Bolt-conn-event-executor-5-thread-1][com.alipay.sofa.jraft.rpc.impl.core.ClientServiceConnectionEventProcessor]: Peer 192.168.5.98:8085 is connected
[2022-05-19 20:41:59,468][INFO][Bolt-default-executor-6-thread-6][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=390, currTerm=389, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:41:59,468][INFO][Bolt-default-executor-6-thread-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=390, currTerm=389, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:41:59,474][INFO][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteResponse from 192.168.5.98:8085, term=389, granted=true.
[2022-05-19 20:41:59,474][INFO][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteResponse from 192.168.5.98:8084, term=389, granted=true.
[2022-05-19 20:41:59,474][INFO][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> start vote and grant vote self, term=389.
[2022-05-19 20:41:59,474][INFO][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> start vote and grant vote self, term=389.
[2022-05-19 20:42:00,477][ERROR][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:00,477][ERROR][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:00,477][WARN][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:00,477][WARN][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:00,492][WARN][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:42:00,492][WARN][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\raft_meta on windows.
[2022-05-19 20:42:00,493][INFO][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=390, votedFor=192.168.5.98:8085, cost time=14 ms
[2022-05-19 20:42:00,493][INFO][JRaft-RPC-Processor-0][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/master/data\raft_meta, term=390, votedFor=192.168.5.98:8084, cost time=14 ms
[2022-05-19 20:42:00,493][INFO][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received RequestVoteRequest from 192.168.5.98:8084, term=390, currTerm=390.
[2022-05-19 20:42:00,493][INFO][Bolt-default-executor-6-thread-9][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received RequestVoteRequest from 192.168.5.98:8085, term=390, currTerm=390.
[2022-05-19 20:42:00,714][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8084> term 390 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:00,714][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 390 start preVote.
[2022-05-19 20:42:01,190][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8085> term 390 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:01,190][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 390 start preVote.
[2022-05-19 20:42:01,718][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:01,718][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:01,825][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 390 start preVote.
[2022-05-19 20:42:02,193][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:02,193][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:02,193][INFO][Bolt-default-executor-6-thread-16][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=391, currTerm=390, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:02,310][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 390 start preVote.
[2022-05-19 20:42:02,827][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:02,827][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:02,828][INFO][Bolt-default-executor-6-thread-1][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=391, currTerm=390, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:02,828][INFO][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteResponse from 192.168.5.98:8085, term=390, granted=true.
[2022-05-19 20:42:02,828][INFO][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> start vote and grant vote self, term=390.
[2022-05-19 20:42:03,199][ERROR][scheduling-1][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:03,199][ERROR][scheduling-1][com.alipay.sofa.jraft.core.CliServiceImpl]: Fail to connect peer 192.168.5.98:8086 to get leader for group TEST2.
java.lang.IllegalStateException: Fail to get leader of group TEST2, Unknown leader, Unknown leader
at com.alipay.sofa.jraft.core.CliServiceImpl.getPeers(CliServiceImpl.java:605)
at com.alipay.sofa.jraft.core.CliServiceImpl.getAlivePeers(CliServiceImpl.java:503)
at com.visec.platform.webstarter.cluster.core.jraft.RaftHandle.getNodeHealthByCluster(RaftHandle.java:119)
at com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks.monitorEndpointHealth(ScheduledRaftTasks.java:334)
at com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks.monitorEndpointStatus(ScheduledRaftTasks.java:260)
at com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks$$FastClassBySpringCGLIB$$a2e2d2aa.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks$$EnhancerBySpringCGLIB$$943a2a93.monitorEndpointStatus()
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:305)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
[2022-05-19 20:42:03,321][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:03,321][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:03,321][INFO][Bolt-default-executor-6-thread-4][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=391, currTerm=390, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:03,321][INFO][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteResponse from 192.168.5.98:8084, term=390, granted=true.
[2022-05-19 20:42:03,322][INFO][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> start vote and grant vote self, term=390.
[2022-05-19 20:42:03,831][ERROR][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:03,831][WARN][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:03,834][WARN][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\raft_meta on windows.
[2022-05-19 20:42:03,834][INFO][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/master/data\raft_meta, term=391, votedFor=192.168.5.98:8084, cost time=3 ms
[2022-05-19 20:42:03,834][INFO][Bolt-default-executor-6-thread-7][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=391, currTerm=391, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:03,834][WARN][JRaft-RPC-Processor-3][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received invalid PreVoteResponse from 192.168.5.98:8085, state not in STATE_FOLLOWER but STATE_CANDIDATE.
[2022-05-19 20:42:04,323][ERROR][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:04,323][WARN][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:04,324][INFO][Bolt-default-executor-6-thread-13][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received RequestVoteRequest from 192.168.5.98:8085, term=391, currTerm=391.
[2022-05-19 20:42:04,325][WARN][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:42:04,325][INFO][JRaft-RPC-Processor-2][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=391, votedFor=192.168.5.98:8085, cost time=3 ms
[2022-05-19 20:42:04,326][INFO][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received RequestVoteRequest from 192.168.5.98:8084, term=391, currTerm=391.
[2022-05-19 20:42:04,326][WARN][JRaft-RPC-Processor-3][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received invalid PreVoteResponse from 192.168.5.98:8084, state not in STATE_FOLLOWER but STATE_CANDIDATE.
[2022-05-19 20:42:04,396][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8085> term 391 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:04,396][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 391 start preVote.
[2022-05-19 20:42:04,563][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8084> term 391 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:04,563][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 391 start preVote.
[2022-05-19 20:42:05,329][ERROR][scheduling-1][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:05,329][ERROR][scheduling-1][com.alipay.sofa.jraft.core.CliServiceImpl]: Fail to connect peer 192.168.5.98:8086 to get leader for group TEST2.
[2022-05-19 20:42:05,399][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:05,399][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:05,432][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 391 start preVote.
[2022-05-19 20:42:05,567][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:05,567][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:05,568][INFO][Bolt-default-executor-6-thread-19][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=392, currTerm=391, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:06,026][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 391 start preVote.
[2022-05-19 20:42:06,433][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:06,434][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:06,434][INFO][Bolt-default-executor-6-thread-16][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=392, currTerm=391, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:06,434][INFO][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteResponse from 192.168.5.98:8084, term=391, granted=true.
[2022-05-19 20:42:06,435][INFO][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> start vote and grant vote self, term=391.
[2022-05-19 20:42:07,027][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:07,027][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:07,028][INFO][Bolt-default-executor-6-thread-1][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=392, currTerm=391, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:07,028][INFO][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteResponse from 192.168.5.98:8085, term=391, granted=true.
[2022-05-19 20:42:07,028][INFO][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> start vote and grant vote self, term=391.
[2022-05-19 20:42:07,437][ERROR][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:07,437][WARN][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:07,440][WARN][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:42:07,440][INFO][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=392, votedFor=192.168.5.98:8085, cost time=3 ms
[2022-05-19 20:42:07,440][INFO][Bolt-default-executor-6-thread-4][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=392, currTerm=392, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:07,440][WARN][JRaft-RPC-Processor-6][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received invalid PreVoteResponse from 192.168.5.98:8084, state not in STATE_FOLLOWER but STATE_CANDIDATE.
[2022-05-19 20:42:07,715][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8085> term 392 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:07,715][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 392 start preVote.
[2022-05-19 20:42:08,031][ERROR][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:08,031][WARN][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:08,034][WARN][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\raft_meta on windows.
[2022-05-19 20:42:08,034][INFO][JRaft-RPC-Processor-5][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/master/data\raft_meta, term=392, votedFor=192.168.5.98:8084, cost time=3 ms
[2022-05-19 20:42:08,034][INFO][Bolt-default-executor-6-thread-5][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received RequestVoteRequest from 192.168.5.98:8085, term=392, currTerm=392.
[2022-05-19 20:42:08,034][WARN][JRaft-RPC-Processor-6][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received invalid PreVoteResponse from 192.168.5.98:8085, state not in STATE_FOLLOWER but STATE_CANDIDATE.
[2022-05-19 20:42:08,336][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8084> term 392 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:08,336][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 392 start preVote.
[2022-05-19 20:42:08,442][ERROR][scheduling-1][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:08,443][ERROR][scheduling-1][com.alipay.sofa.jraft.core.CliServiceImpl]: Fail to connect peer 192.168.5.98:8086 to get leader for group TEST2.
[2022-05-19 20:42:08,717][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:08,717][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:08,717][INFO][Bolt-default-executor-6-thread-11][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received RequestVoteRequest from 192.168.5.98:8084, term=392, currTerm=392.
[2022-05-19 20:42:08,718][WARN][JRaft-RPC-Processor-7][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received invalid RequestVoteResponse from 192.168.5.98:8084, state not in STATE_CANDIDATE but STATE_FOLLOWER.
[2022-05-19 20:42:09,338][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:09,338][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:09,339][INFO][Bolt-default-executor-6-thread-11][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=393, currTerm=392, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:09,339][WARN][JRaft-RPC-Processor-7][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received invalid RequestVoteResponse from 192.168.5.98:8085, state not in STATE_CANDIDATE but STATE_FOLLOWER.
[2022-05-19 20:42:09,339][INFO][Bolt-default-executor-6-thread-15][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=393, currTerm=392, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:42:09,341][INFO][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteResponse from 192.168.5.98:8084, term=392, granted=true.
[2022-05-19 20:42:09,341][INFO][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteResponse from 192.168.5.98:8085, term=392, granted=true.
[2022-05-19 20:42:09,341][INFO][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> start vote and grant vote self, term=392.
[2022-05-19 20:42:09,341][INFO][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> start vote and grant vote self, term=392.
[2022-05-19 20:42:10,343][ERROR][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:10,343][ERROR][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:10,343][WARN][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:10,343][WARN][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:42:10,346][WARN][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:42:10,346][WARN][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\raft_meta on windows.
[2022-05-19 20:42:10,346][INFO][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=393, votedFor=192.168.5.98:8085, cost time=3 ms
[2022-05-19 20:42:10,346][INFO][JRaft-RPC-Processor-8][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/master/data\raft_meta, term=393, votedFor=192.168.5.98:8084, cost time=3 ms
[2022-05-19 20:42:10,346][INFO][Bolt-default-executor-6-thread-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received RequestVoteRequest from 192.168.5.98:8084, term=393, currTerm=393.
[2022-05-19 20:42:10,346][INFO][Bolt-default-executor-6-thread-16][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received RequestVoteRequest from 192.168.5.98:8085, term=393, currTerm=393.
[2022-05-19 20:42:10,915][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8085> term 393 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:10,915][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 393 start preVote.
[2022-05-19 20:42:11,187][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8084> term 393 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:42:11,187][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 393 start preVote.
[2022-05-19 20:42:11,348][ERROR][scheduling-1][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:42:11,348][ERROR][scheduling-1][com.alipay.sofa.jraft.core.CliServiceImpl]: Fail to connect peer 192.168.5.98:8086 to get leader for group TEST2.
2、这个是等待一些时间后,B机器自己连接成功,A机器并没有进行操作开启服务
[2022-05-19 20:43:32,798][ERROR][JRaft-RPC-Processor-18][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:32,798][WARN][JRaft-RPC-Processor-18][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:43:32,801][WARN][JRaft-RPC-Processor-18][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\raft_meta on windows.
[2022-05-19 20:43:32,801][INFO][JRaft-RPC-Processor-18][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/master/data\raft_meta, term=420, votedFor=192.168.5.98:8084, cost time=3 ms
[2022-05-19 20:43:32,876][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Candidate node <TEST2/192.168.5.98:8084> term 420 steps down when election reaching vote timeout: fail to get quorum vote-granted.
[2022-05-19 20:43:32,876][INFO][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> term 420 start preVote.
[2022-05-19 20:43:33,172][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:33,172][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:43:33,172][INFO][Bolt-default-executor-6-thread-8][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received RequestVoteRequest from 192.168.5.98:8084, term=420, currTerm=419.
[2022-05-19 20:43:33,175][WARN][Bolt-default-executor-6-thread-8][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:43:33,175][INFO][Bolt-default-executor-6-thread-8][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=420, votedFor=0.0.0.0:0, cost time=3 ms
[2022-05-19 20:43:33,176][WARN][Bolt-default-executor-6-thread-8][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:43:33,176][INFO][Bolt-default-executor-6-thread-8][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=420, votedFor=192.168.5.98:8084, cost time=1 ms
[2022-05-19 20:43:33,890][ERROR][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:33,890][WARN][JRaft-VoteTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:43:33,891][INFO][Bolt-default-executor-6-thread-2][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=420, currTerm=420, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:43:33,891][WARN][JRaft-RPC-Processor-19][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received invalid RequestVoteResponse from 192.168.5.98:8085, state not in STATE_CANDIDATE but STATE_FOLLOWER.
[2022-05-19 20:43:33,891][INFO][Bolt-default-executor-6-thread-3][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received PreVoteRequest from 192.168.5.98:8084, term=421, currTerm=420, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:43:33,891][WARN][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received invalid PreVoteResponse from 192.168.5.98:8084, term=419, currTerm=420.
[2022-05-19 20:43:33,892][INFO][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteResponse from 192.168.5.98:8085, term=420, granted=true.
[2022-05-19 20:43:33,892][INFO][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> start vote and grant vote self, term=420.
[2022-05-19 20:43:34,185][ERROR][scheduling-1][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:34,185][ERROR][scheduling-1][com.alipay.sofa.jraft.core.CliServiceImpl]: Fail to connect peer 192.168.5.98:8086 to get leader for group TEST2.
[2022-05-19 20:43:34,201][INFO][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> term 420 start preVote.
[2022-05-19 20:43:34,903][ERROR][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:34,903][WARN][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:43:34,905][WARN][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\raft_meta on windows.
[2022-05-19 20:43:34,905][INFO][JRaft-RPC-Processor-20][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/master/data\raft_meta, term=421, votedFor=192.168.5.98:8084, cost time=2 ms
[2022-05-19 20:43:35,202][ERROR][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:35,202][WARN][JRaft-ElectionTimer-<TEST2/192.168.5.98:8085>0][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> channel init failed, address=192.168.5.98:8086.
[2022-05-19 20:43:35,202][INFO][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received RequestVoteRequest from 192.168.5.98:8084, term=421, currTerm=420.
[2022-05-19 20:43:35,203][INFO][Bolt-default-executor-6-thread-1][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> received PreVoteRequest from 192.168.5.98:8085, term=421, currTerm=421, granted=true, requestLastLogId=LogId [index=109, term=389], lastLogId=LogId [index=109, term=389].
[2022-05-19 20:43:35,209][WARN][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:43:35,209][INFO][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=421, votedFor=0.0.0.0:0, cost time=7 ms
[2022-05-19 20:43:35,211][WARN][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\raft_meta on windows.
[2022-05-19 20:43:35,211][INFO][Bolt-default-executor-6-thread-10][com.alipay.sofa.jraft.storage.impl.LocalRaftMetaStorage]: Save raft meta, path=/database/jraft/backup/data\raft_meta, term=421, votedFor=192.168.5.98:8084, cost time=2 ms
[2022-05-19 20:43:35,212][WARN][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8085> received invalid PreVoteResponse from 192.168.5.98:8084, term=420, currTerm=421.
[2022-05-19 20:43:35,212][INFO][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.core.NodeImpl]: Node <TEST2/192.168.5.98:8084> become leader of group, term=421, conf=192.168.5.98:8084,192.168.5.98:8085,192.168.5.98:8086, oldConf=.
[2022-05-19 20:43:36,217][ERROR][scheduling-1][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:36,217][ERROR][scheduling-1][com.alipay.sofa.jraft.core.CliServiceImpl]: Fail to connect peer 192.168.5.98:8086 to get leader for group TEST2.
[2022-05-19 20:43:36,217][ERROR][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.rpc.impl.AbstractClientService]: Fail to connect 192.168.5.98:8086, remoting exception: com.alipay.remoting.exception.RemotingException: Create connection failed. The address is 192.168.5.98:8086.
[2022-05-19 20:43:36,217][ERROR][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.core.Replicator]: Fail to init sending channel to 192.168.5.98:8086.
[2022-05-19 20:43:36,217][ERROR][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to start replicator to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:36,217][ERROR][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.core.NodeImpl]: Fail to add a replicator, peer=192.168.5.98:8086.
[2022-05-19 20:43:36,218][INFO][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.core.Replicator]: Replicator=Replicator [state=Created, statInfo=<running=null, firstLogIndex=0, lastLogIncluded=0, lastLogIndex=0, lastTermIncluded=0>, peerId=192.168.5.98:8085, type=Follower]@192.168.5.98:8085 is started
[2022-05-19 20:43:36,222][WARN][Rpc-netty-server-worker-1-thread-2][com.alipay.sofa.jraft.rpc.impl.BoltRaftRpcFactory]: JRaft SET bolt.rpc.dispatch-msg-list-in-default-executor to be false for replicator pipeline optimistic.
[2022-05-19 20:43:36,225][INFO][JRaft-RPC-Processor-21][com.alipay.sofa.jraft.util.Recyclers]: -Djraft.recyclers.maxCapacityPerThread: 4096.
[2022-05-19 20:43:36,227][INFO][JRaft-FSMCaller-Disruptor-0][com.alipay.sofa.jraft.core.StateMachineAdapter]: onStartFollowing: LeaderChangeContext [leaderId=192.168.5.98:8084, term=421, status=Status[ENEWLEADER<10011>: Follower receives message from new leader with the same term.]].
Hibernate: select sysraftclu0_.c_id as c_id1_10_0_, sysraftclu0_.c_balance_mode as c_balanc2_10_0_, sysraftclu0_.c_cluster_mode as c_cluste3_10_0_, sysraftclu0_.c_cluster_status as c_cluste4_10_0_, sysraftclu0_.c_endpoints as c_endpoi5_10_0_, sysraftclu0_.c_endpoints_vip as c_endpoi6_10_0_, sysraftclu0_.c_group_id as c_group_7_10_0_, sysraftclu0_.c_ip as c_ip8_10_0_, sysraftclu0_.c_joined as c_joined9_10_0_, sysraftclu0_.c_license_port as c_licen10_10_0_, sysraftclu0_.c_master as c_maste11_10_0_, sysraftclu0_.c_node as c_node12_10_0_, sysraftclu0_.c_node2 as c_node13_10_0_, sysraftclu0_.c_other_endpoints_vip as c_other14_10_0_, sysraftclu0_.c_port as c_port15_10_0_, sysraftclu0_.c_status as c_statu16_10_0_, sysraftclu0_.c_sync_status as c_sync_17_10_0_, sysraftclu0_.c_vip_iface as c_vip_i18_10_0_ from sys_raft_cluster sysraftclu0_ where sysraftclu0_.c_id=?
[2022-05-19 20:43:36,239][INFO][scheduling-1][com.visec.example.cluster.SubscriberImpl]: 业务系统收到endpoint发来的消息:{"newValue":true,"noticeType":"HEALTH","oldValue":false}
[2022-05-19 20:43:36,240][INFO][JRaft-FSMCaller-Disruptor-0][com.alipay.sofa.jraft.core.StateMachineAdapter]: onLeaderStart: term=421.
[2022-05-19 20:43:36,240][INFO][scheduling-1][com.visec.example.cluster.SubscriberImpl]: 业务系统收到endpoint发来的消息:{"newValue":true,"noticeType":"LEADER","oldValue":false}
[2022-05-19 20:43:36,240][INFO][scheduling-1][com.visec.example.cluster.SubscriberImpl]: 启动业务服务开始-----
[2022-05-19 20:43:36,240][INFO][scheduling-1][com.visec.example.cluster.SubscriberImpl]: 启动业务服务完毕-----
[2022-05-19 20:43:36,241][INFO][scheduling-1][com.visec.platform.webstarter.cluster.web.service.RaftService]: fullVersion==1.1.1
[2022-05-19 20:43:36,241][INFO][scheduling-1][com.visec.platform.webstarter.cluster.web.service.RaftService]: modulesList==[]
[2022-05-19 20:43:36,244][INFO][Bolt-conn-event-executor-5-thread-1][com.alipay.sofa.jraft.rpc.impl.core.ClientServiceConnectionEventProcessor]: Peer 192.168.5.98:8084 is connected
[2022-05-19 20:43:36,319][INFO][JRaft-FSMCaller-Disruptor-0][com.visec.platform.webstarter.cluster.core.jraft.RaftStateMachine]: Added value, logIndex=111
[2022-05-19 20:43:36,321][INFO][scheduling-1][com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks]: 更新验证信息,Optional.empty
[2022-05-19 20:43:36,321][INFO][scheduling-1][com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks]: 验证信息已同步到各个节点
[2022-05-19 20:43:36,328][INFO][JRaft-FSMCaller-Disruptor-0][com.visec.platform.webstarter.cluster.core.jraft.RaftStateMachine]: Added value, logIndex=111
[2022-05-19 20:43:36,329][INFO][scheduling-1][com.visec.platform.webstarter.cluster.web.service.RaftService]: fullVersion==1.1.1
[2022-05-19 20:43:36,329][INFO][scheduling-1][com.visec.platform.webstarter.cluster.web.service.RaftService]: modulesList==[]
[2022-05-19 20:43:36,329][INFO][scheduling-1][com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks]: 版本信息, leader[1.1.1], 本地[1.1.1]
[2022-05-19 20:43:36,329][INFO][scheduling-1][com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks]: 模块信息, leader[], 本地[]
[2022-05-19 20:43:36,330][INFO][scheduling-1][com.visec.platform.webstarter.cluster.core.tasks.ScheduledRaftTasks]: 验证通过,启动jraft
Hibernate: update sys_raft_cluster set c_balance_mode=?, c_cluster_mode=?, c_cluster_status=?, c_endpoints=?, c_endpoints_vip=?, c_group_id=?, c_ip=?, c_joined=?, c_license_port=?, c_master=?, c_node=?, c_node2=?, c_other_endpoints_vip=?, c_port=?, c_status=?, c_sync_status=?, c_vip_iface=? where c_id=?
[2022-05-19 20:43:36,388][INFO][scheduling-1][com.visec.example.cluster.SubscriberImpl]: 业务系统收到cluster发来的消息:[{"newValue":true,"endpoint":"192.168.5.98:8084","groupId":"TEST2","self":true,"noticeType":"LEADER","oldValue":false}, {"newValue":true,"endpoint":"192.168.5.98:8084","groupId":"TEST2","self":true,"noticeType":"HEALTH","oldValue":false}, {"newValue":true,"endpoint":"192.168.5.98:8085","groupId":"TEST2","self":false,"noticeType":"HEALTH","oldValue":false}]
[2022-05-19 20:43:36,730][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:37,243][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:37,758][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:38,272][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:38,784][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:39,297][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:39,811][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:40,324][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:40,835][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:41,349][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:41,862][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:42,376][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:42,890][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:43,406][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:43,920][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:44,430][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:44,944][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:45,458][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:45,961][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:46,464][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:46,980][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:47,483][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:47,997][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:48,500][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:49,001][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:49,504][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:50,007][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:50,148][INFO][JRaft-Closure-Executor-11][com.visec.platform.webstarter.cluster.core.jraft.RaftStateMachine]: 开启快照!
[2022-05-19 20:43:50,153][WARN][JRaft-Closure-Executor-10][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\snapshot\temp on windows.
[2022-05-19 20:43:50,153][INFO][JRaft-Closure-Executor-10][com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotStorage]: Deleting snapshot /database/jraft/master/data\snapshot\snapshot_111.
[2022-05-19 20:43:50,153][INFO][JRaft-Closure-Executor-10][com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotStorage]: Renaming /database/jraft/master/data\snapshot\temp to /database/jraft/master/data\snapshot\snapshot_111.
[2022-05-19 20:43:50,154][WARN][JRaft-Closure-Executor-10][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\master\data\snapshot on windows.
[2022-05-19 20:43:50,154][INFO][JRaft-Closure-Executor-10][com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotStorage]: Deleting snapshot /database/jraft/master/data\snapshot\snapshot_109.
[2022-05-19 20:43:50,158][INFO][JRaft-Closure-Executor-10][com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage]: Truncated prefix logs in data path: /database/jraft/master/data\log from log index 110 to 110, cost 0 ms.
[2022-05-19 20:43:50,508][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:51,009][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:51,511][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:52,013][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:52,514][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:53,017][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:53,518][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:54,019][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:54,521][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:55,022][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.
[2022-05-19 20:43:55,376][INFO][JRaft-Closure-Executor-10][com.visec.platform.webstarter.cluster.core.jraft.RaftStateMachine]: 开启快照!
[2022-05-19 20:43:55,379][WARN][JRaft-Closure-Executor-11][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\snapshot\temp on windows.
[2022-05-19 20:43:55,379][INFO][JRaft-Closure-Executor-11][com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotStorage]: Deleting snapshot /database/jraft/backup/data\snapshot\snapshot_111.
[2022-05-19 20:43:55,379][INFO][JRaft-Closure-Executor-11][com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotStorage]: Renaming /database/jraft/backup/data\snapshot\temp to /database/jraft/backup/data\snapshot\snapshot_111.
[2022-05-19 20:43:55,379][WARN][JRaft-Closure-Executor-11][com.alipay.sofa.jraft.util.Utils]: Unable to fsync directory \database\jraft\backup\data\snapshot on windows.
[2022-05-19 20:43:55,379][INFO][JRaft-Closure-Executor-11][com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotStorage]: Deleting snapshot /database/jraft/backup/data\snapshot\snapshot_109.
[2022-05-19 20:43:55,381][INFO][JRaft-Closure-Executor-11][com.alipay.sofa.jraft.storage.impl.RocksDBLogStorage]: Truncated prefix logs in data path: /database/jraft/backup/data\log from log index 110 to 110, cost 0 ms.
[2022-05-19 20:43:55,523][ERROR][JRaft-StepDownTimer-<TEST2/192.168.5.98:8084>0][com.alipay.sofa.jraft.core.ReplicatorGroupImpl]: Fail to check replicator connection to peer=192.168.5.98:8086, replicatorType=Follower.