straumat
Performance after block 250k-300k
It appears that the more records that are inserted the slower the process get's, performance seemed to drop off sharply after block 250k-300k. Just looking at the GUI it appears to be the "saving block" step that takes the longest. Are there any tweaks or changes I can make ? The I'm using has 20 cores, 96 GB pf memory and has all the docker containers (straumat/blockchain2graph-bitcoin-neo4j:3.0) running locally.
Sadly no :( i did not find any way to make it faster... it's more of a neo4j issue I think :(
Could it be the indexes slowing down writes (as they need to be updated) as mentioned here: https://stackoverflow.com/questions/18212860/neo4j-slow-savin] ? Are they needed for the import or can they be deleted and re-created after the import ?
Yes, indexes slow down the writes but to make analysis, before saving a new block, I link all transactions to their outcomes so I guess it will make things very slow to retrieve all the output transactions before saying. You can try to delete them and recreate them after if you want to try.