[R4R] use priority lock for blockchain

[forcodedancing]

Description

This pr tries to improve block importing by using priority lock for blockchain, which will give block insertion higher priority than others.

Rationale

During block insertion there are other subroutines also try to access the ongoing blockchain, which could slowdown the main routine of block insertion.

Result

• Before

Dropped 527 nodes (cum <= 0.01mins)
Showing top 40 nodes out of 126
      flat  flat%   sum%        cum   cum%
  1.99mins 87.30% 87.30%   1.99mins 87.30%  sync.(*Mutex).Unlock (partial-inline)
  0.27mins 11.94% 99.24%   1.91mins 83.79%  sync.(*RWMutex).Unlock
  0.02mins  0.76%   100%   0.02mins  0.76%  sync.(*RWMutex).RUnlock (inline)
         0     0%   100%   0.02mins  0.83%  github.com/VictoriaMetrics/fastcache.(*Cache).Set
         0     0%   100%   0.02mins  0.83%  github.com/VictoriaMetrics/fastcache.(*bucket).Set
         0     0%   100%   0.77mins 33.82%  github.com/ethereum/go-ethereum/core.(*BlockChain).InsertChain
         0     0%   100%   0.02mins  0.92%  github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain
         0     0%   100%   0.01mins  0.65%  github.com/ethereum/go-ethereum/core.(*BlockChain).writeBlockWithState
         0     0%   100%   0.01mins  0.65%  github.com/ethereum/go-ethereum/core.(*BlockChain).writeHeadBlock

• After

Dropped 525 nodes (cum <= 0.01mins)
Showing top 40 nodes out of 125
      flat  flat%   sum%        cum   cum%
  1.35mins 59.00% 59.00%   1.35mins 59.00%  sync.(*Mutex).Unlock (partial-inline)
  0.92mins 40.09% 99.09%   1.95mins 85.46%  sync.(*RWMutex).Unlock
  0.02mins  0.91%   100%   0.02mins  0.91%  sync.(*RWMutex).RUnlock (inline)
         0     0%   100%   0.02mins  0.77%  github.com/VictoriaMetrics/fastcache.(*Cache).Set
         0     0%   100%   0.02mins  0.77%  github.com/VictoriaMetrics/fastcache.(*bucket).Set
         0     0%   100%   0.02mins  1.05%  github.com/ethereum/go-ethereum/common/prlock.(*Prlock).LockHigh
         0     0%   100%   0.75mins 32.90%  github.com/ethereum/go-ethereum/common/prlock.(*Prlock).UnlockHigh (inline)
         0     0%   100%   0.80mins 35.07%  github.com/ethereum/go-ethereum/core.(*BlockChain).InsertChain
         0     0%   100%   0.03mins  1.13%  github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain
         0     0%   100%   0.02mins  0.84%  github.com/ethereum/go-ethereum/core.(*BlockChain).writeBlockWithState
         0     0%   100%   0.02mins  0.84%  github.com/ethereum/go-ethereum/core.(*BlockChain).writeHeadBlock

We can see for using priority lock, InsertChain block time increased, from 0.77 to 0.8. These two nodes did not response to any other requests/workload. There is a chance that priority can work better when there are other workloads. AccountsIntermediateRoot block time has beed decreased.

Changes

Notable changes:

• add priority lock for block insertion

[xiyuewang3]

Great changes