ethereumjs-monorepo icon indicating copy to clipboard operation
ethereumjs-monorepo copied to clipboard

Master tracker for snap sync

Open am1r021 opened this issue 2 years ago • 2 comments

See #2100 and #1848 for work done up to this point and resources on snap sync.

Starting with a new master tracker ticket for continuing work on snap sync. We currently have the ability to perform a static sync. Next step is to support a moving sync.

  • [x] merge #2623
    • [x] Refactor encoding module with a focus on function naming and deduplication
    • [x] Unit testing
  • [x] Simulated testing #2623
  • [x] Integration: #3028 #3031 #3033
  • [ ] Trie Healing #3332
  • [ ] Devnet and Mainnet testing
  • [ ] Benchmarking
  • [ ] Verkle snap sync planning and implementation as followup

cc: @g11tech @jochem-brouwer @holgerd77

am1r021 avatar Jul 10 '23 20:07 am1r021

You should benchmark the differences in this instantation of the trie using leveldb versus just removing that leveldb from the trie constructor (which would then default to using the MapDB from our util library. My suspicion is that the map DB is slightly faster.

acolytec3 avatar Jul 25 '23 18:07 acolytec3

Okay, so here's some benchmarking for mapdb vs leveldb. Here's my benchmarking script:

import { Trie } from '@ethereumjs/trie'
import { randomBytes } from 'crypto'
//@ts-ignore
import * as bench from 'micro-bmark'

import { LevelDB } from './level'
const { mark, compare, run } = bench // Or, use as such
const trieOps = async (trie: Trie) => {
  const x = new Uint8Array(300)
  for await (const _l of x) {
    await trie.put(randomBytes(32), randomBytes(32))
  }
}
run(async () => {
  await mark('mapdb', 100, async () => trieOps(new Trie()))
  await mark('level', 100, async () => trieOps(new Trie({ db: new LevelDB() })))
})

I'm using micro-bmark as my benchmark tool here so you'll have to install that. But, the results seem reasonably convincing to me for switching from the leveldb to mapdb:

-------
Benchmarking
mapdb x 16 ops/sec @ 60ms/op ± 1.42% (min: 52ms, max: 89ms)
level x 14 ops/sec @ 68ms/op ± 1.48% (min: 61ms, max: 104ms)

-------
Benchmarking
mapdb x 18 ops/sec @ 53ms/op ± 1.38% (min: 49ms, max: 76ms)
level x 15 ops/sec @ 63ms/op ± 1.45% (min: 58ms, max: 95ms)

acolytec3 avatar Jul 25 '23 18:07 acolytec3