aptos-ait2 icon indicating copy to clipboard operation
aptos-ait2 copied to clipboard

Published 20 hours ago • verified publisher icon aptos-labs

Node lagging and no proposals being sent for a long time

Open guruchaithanya opened this issue 2 years ago • 2 comments

Node config: 8 vCPUs, 32 GB RAM, 640 GB SSH. AWS VPS, Location: India Commit: testnet_f5d8013b0a1851da8e078394d83130d3adaf7670 Node address: 0x6c76c5874f103603534a2e9bf8dcd0b54f0ed767258820193444b7419beddb22 Installation: Source Code on Ubuntu VPS Location: India

My node proposal count is dismal (sometimes remains 0 zero) and stops increasing as node goes out or remain out of sync.

I have attached the output logs for the scenario where the proposal count never increases above 0 zero.

No proposals observed.

Relevant Timestamps: 2022-07-17T 10:58:16: Restarted the aptos node 2022-07-17T 11:51:31: At this point, the sync state never recovers 2022-07-17T 11:55:23: No more peers available error starts 2022-07-17T12:26:12: End of logs captured, still node is out of sync.

So even after 1 and half hour the node never syncs and never sends a proposal.

Please find the root cause and resolve this issue.

Node logs location (31 MB zip file): https://drive.google.com/drive/folders/1viE0nFXWmIdtHKB1002pBVAvC-7vUDul?usp=sharing

IssueNodeMetric 2.log IssueNodeMetrics 1.log IssueNodeMetrics 3.txt

Network "nload" output: image

ubuntu@ip-172-26-12-10:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/root 621G 40G 581G 7% / devtmpfs 16G 0 16G 0% /dev tmpfs 16G 0 16G 0% /dev/shm tmpfs 3.2G 908K 3.2G 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 16G 0 16G 0% /sys/fs/cgroup /dev/loop0 29M 29M 0 100% /snap/amazon-ssm-agent/2012 /dev/loop1 26M 26M 0 100% /snap/amazon-ssm-agent/5656 /dev/loop2 114M 114M 0 100% /snap/core/13425 /dev/loop3 55M 55M 0 100% /snap/core18/1880 /dev/loop4 56M 56M 0 100% /snap/core18/2409 /dev/loop5 62M 62M 0 100% /snap/core20/1518 /dev/loop7 72M 72M 0 100% /snap/lxd/16100 /dev/loop6 103M 103M 0 100% /snap/lxd/23270 tmpfs 3.2G 0 3.2G 0% /run/user/1000

Top command output: image

guruchaithanya avatar Jul 17 '22 12:07 guruchaithanya

Are you sure you build the source code from the latest of the testnet branch? could you please share your git log output?

sherry-x avatar Jul 19 '22 06:07 sherry-x

Hi Sherry, Yes I am using latest recommended testnet commit f5d8013b0a1851da8e078394d83130d3adaf7670.

PFA git log output.

gitlogoutput.log

Thanks, Guru

guruchaithanya avatar Jul 20 '22 07:07 guruchaithanya