bitcoingraph
bitcoingraph copied to clipboard
behas
Requirements, time and neo4j format
I will soon try this tool but I would like to understand in advance three things:
- What are the requirements in terms of CPU, Disk and RAM, to run the whole thing?
- How much time would it get to fully sync the blockchain and export in Neo4j?
- In the Neo4j graph, what do nodes and edges represent? Addresses and transactions? Or it's a multi-graph with transactions as nodes and inputs/outputs as edges?
Expect at least 256 gigabytes of disk space, give it a few days to process on a m1.medium instance.
I just did blocks 0-677000, i.e. up to March 2021. The chain itself in $HOME/.bitcoind (up to current height) takes 530 GiB, the csv files for 0-677000 another 669 GiB, and then neo4j estimates it will need yet another ~590 GiB for the database.
Fully syncing the blockchain depends on when you do it (it keeps growing every ten minutes) and the speed of your internet connection. Producing the csv files depends on the speed of your disk(s), on your CPU and to a lesser extent on available RAM; on my AMD 5600G and a WD40 HDD it took about 40 hours. You can probably speed it up by keeping source and destination (blockchain->csv and csv->neo4j) on different disks and by not using copy on write on the target filesystem. Importing into neo4j will probably need 32 GB RAM, perhaps even more. When you start the import you get some suggestions for heap and other memory. Make very sure that you don't use swap.
Now I know. It took 7 years, but totally worth the wait 😉. Jokes aside, I remember having this need and back then requirements were of course a bit more forgiving. Not on this topics these days but it's nice to see there's still people investing their time in such tools.