cybernode-archive icon indicating copy to clipboard operation
cybernode-archive copied to clipboard
verified publisher icon cyberFund

Create optimized base container for our containers with blockchain systems

Open cyborgshead opened this issue 8 years ago • 10 comments

cyborgshead avatar May 23 '17 09:05 cyborgshead

@abitrolly research this please https://github.com/cusspvz/autotune.docker

cyborgshead avatar May 23 '17 09:05 cyborgshead

@abitrolly check this http://phusion.github.io/baseimage-docker/

cyborgshead avatar May 23 '17 09:05 cyborgshead

Let me clarify - this setup is for full blockchain nodes?

I need to get more info about what type of workload to expect from these nodes? Optimizing network stack for 300.000 concurrent connections may be not as worthy as more CPU cycles to query blockchain itself.

autotune looks reasonable. Let's setup monitoring first to check that those IPv4 settings really matter.

abitrolly avatar May 23 '17 11:05 abitrolly

I am checking what are the best practices to keep containers up to date with security updates in base images - https://stackoverflow.com/questions/26423515/how-to-automatically-update-your-docker-containers-if-base-images-are-updated

And so we also need to decide what would be the base image. 16.04 LTS or 17.04 or something more exotic? I am comfortable with 17.04 though.

abitrolly avatar May 23 '17 11:05 abitrolly

@abitrolly LTS version, and yes, full blockchain nodes, but directories with chain data should be mounted

cyborgshead avatar May 23 '17 12:05 cyborgshead

Where are those storage mounts physically and logically located?

abitrolly avatar May 23 '17 14:05 abitrolly

Meeting notes. Blockchain data is mounted to /data point inside container and physically located on the host that runs the full node container for now.

abitrolly avatar May 23 '17 15:05 abitrolly

Need monitoring to add current nodes to it to gather requirements. Variants:

Digital Ocean Monitoring

PROS: Easy to install. Graphs are inside DO control panel, easily accessible by everyone with login. CONS: Participation in their data collection policy is not an opt-in. Not clear if the monitoring will work with hardware deployed nodes. https://www.digitalocean.com/community/tutorials/an-introduction-to-digitalocean-monitoring

abitrolly avatar May 27 '17 05:05 abitrolly

Metrics that is needed right now:

  • [ ] bandwidth usage by month - download/upload rate - peaks and their characteristics
  • [ ] traffic usage by month - download/upload in GB
  • [ ] most CPU/memory hungry process/job by month
  • [ ] disk io bottlenecks and close to bottleneck conditions
  • [ ] API/service request waiting time and characteristics of those requests

abitrolly avatar May 27 '17 06:05 abitrolly

Metrics specific to blockchain workload:

  • [ ] size of blockchain database on disk
  • [ ] size of database indexes in memory

abitrolly avatar May 27 '17 06:05 abitrolly