fabric-samples icon indicating copy to clipboard operation
fabric-samples copied to clipboard

Orderer crash when join, unjoin, join with same channel name

Open varadgit opened this issue 3 years ago • 0 comments

I was using the test network from fabric samples and when I did the following sequence of steps which caused the orderer to crash

./network.sh up createChannel -ca -c mychannel5 unjoin the channel - I can see from the osnadmin command that the channel is gone

./network.sh up createChannel -ca -c mychannel5

Orderer crashes with the following

2022-02-24 18:27:36.856 UTC 03a6 INFO [blkstorage] newBlockfileMgr -> Getting block information from block storage
2022-02-24 18:27:36.880 UTC 03a7 INFO [orderer.consensus.etcdraft] HandleChain -> EvictionSuspicion not set, defaulting to 10m0s
2022-02-24 18:27:36.881 UTC 03a8 INFO [orderer.consensus.etcdraft] HandleChain -> Without system channel: after eviction Registrar.SwitchToFollower will be called
2022-02-24 18:27:36.881 UTC 03a9 INFO [orderer.consensus.etcdraft] createOrReadWAL -> Found WAL data at path '/var/hyperledger/production/orderer/etcdraft/wal/mychannel5', replaying it channel=mychannel5 node=1
2022-02-24 18:27:36.883 UTC 03aa INFO [orderer.commmon.multichannel] createAsMember -> Joining channel: {mychannel5  consenter active 1}
2022-02-24 18:27:36.889 UTC 03ab INFO [orderer.consensus.etcdraft] Start -> Starting Raft node channel=mychannel5 node=1
2022-02-24 18:27:36.889 UTC 03ac INFO [orderer.common.cluster] Configure -> Entering, channel: mychannel5, nodes: []
2022-02-24 18:27:36.889 UTC 03ad INFO [orderer.common.cluster] Configure -> Exiting
2022-02-24 18:27:36.889 UTC 03ae INFO [orderer.consensus.etcdraft] start -> Restarting raft node channel=mychannel5 node=1
2022-02-24 18:27:36.890 UTC 03af INFO [orderer.consensus.etcdraft] becomeFollower -> 1 became follower at term 2 channel=mychannel5 node=1
2022-02-24 18:27:36.890 UTC 03b0 INFO [orderer.consensus.etcdraft] newRaft -> newRaft 1 [peers: [], term: 2, commit: 4, applied: 0, lastindex: 4, lastterm: 2] channel=mychannel5 node=1
2022-02-24 18:27:36.894 UTC 03b1 INFO [orderer.consensus.etcdraft] apply -> Applied config change to add node 1, current nodes in channel: [1] channel=mychannel5 node=1
2022-02-24 18:27:36.894 UTC 03b2 INFO [orderer.consensus.etcdraft] writeBlock -> Writing block [1] (Raft index: 3) to ledger channel=mychannel5 node=1
2022-02-24 18:27:36.907 UTC 03b3 PANI [orderer.commmon.multichannel] commitBlock -> [channel: mychannel5] Could not append block: unexpected Previous block hash. Expected PreviousHash = [2c7c76dedc0a97465431b1c404b44b586b355f28091ec26da5f519b57eaa07cf], PreviousHash referred in the latest block= [eae8b0fd832dada16ab08cae33de74b01f2ec5956cee6c27969bf960fc4f59bc]
panic: [channel: mychannel5] Could not append block: unexpected Previous block hash. Expected PreviousHash = [2c7c76dedc0a97465431b1c404b44b586b355f28091ec26da5f519b57eaa07cf], PreviousHash referred in the latest block= [eae8b0fd832dada16ab08cae33de74b01f2ec5956cee6c27969bf960fc4f59bc]

goroutine 11774 [running]:
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000548000, {0x0, 0x0, 0x0})
	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:234 +0x499
go.uber.org/zap.(*SugaredLogger).log(0xc0001a8168, 0x4, {0xf60723, 0x1}, {0xc00015f530, 0x10000000046d41a, 0x1}, {0x0, 0x0, 0x0})
	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0x13b
go.uber.org/zap.(*SugaredLogger).Panicf(...)
	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159
github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(...)
	/go/src/github.com/hyperledger/fabric/common/flogging/zap.go:74
github.com/hyperledger/fabric/orderer/common/multichannel.(*BlockWriter).commitBlock(0xc00125a540, {0xc00110fc36, 0x7, 0x7})
	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/blockwriter.go:203 +0x159
github.com/hyperledger/fabric/orderer/common/multichannel.(*BlockWriter).WriteBlockSync(0xc000e7ea00, 0xc00123b2d0, {0xc00110fc36, 0x113, 0x0})
	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/blockwriter.go:192 +0xeb
github.com/hyperledger/fabric/orderer/common/multichannel.(*BlockWriter).WriteConfigBlock(0xc00125a540, 0xc000f4e450, {0xc00110fc36, 0x7, 0x7})
	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/blockwriter.go:159 +0x81f
github.com/hyperledger/fabric/orderer/consensus/etcdraft.(*Chain).writeConfigBlock(0xc0000bcb00, 0x0, 0x3)
	/go/src/github.com/hyperledger/fabric/orderer/consensus/etcdraft/chain.go:1259 +0x303
github.com/hyperledger/fabric/orderer/consensus/etcdraft.(*Chain).writeBlock(0xc0000bcb00, 0xc000cf5940, 0x3)
	/go/src/github.com/hyperledger/fabric/orderer/consensus/etcdraft/chain.go:873 +0x30d
github.com/hyperledger/fabric/orderer/consensus/etcdraft.(*Chain).apply(0xc0000bcb00, {0xc001200ac8, 0x4, 0x0})
	/go/src/github.com/hyperledger/fabric/orderer/consensus/etcdraft/chain.go:1077 +0x2d9
github.com/hyperledger/fabric/orderer/consensus/etcdraft.(*Chain).run(0xc0000bcb00)
	/go/src/github.com/hyperledger/fabric/orderer/consensus/etcdraft/chain.go:781 +0xaf3
created by github.com/hyperledger/fabric/orderer/consensus/etcdraft.(*Chain).Start
	/go/src/github.com/hyperledger/fabric/orderer/consensus/etcdraft/chain.go:368 +0x2af

Is this a potential Fabric issue with orderers or samples? If this turns out to be an orderer issues I can open the issue on Fabric repo.

varadgit avatar Feb 25 '22 13:02 varadgit