guardian-mainnet-guide icon indicating copy to clipboard operation
guardian-mainnet-guide copied to clipboard

Syncing issues with Theta Guardian node on Ubuntu VPS

Open Krylltrade opened this issue 4 years ago • 9 comments

I followed the guide and the Theta node started to sync. It has now been syncing for approximately 5 hours (guide states should be completed in less than 10 minutes). The VPS specifications is in accordance to, or better than the required specs for running a node.

Can anyone help me sort out this issue?

Krylltrade avatar Jan 09 '21 21:01 Krylltrade

Note, running HTOP command it currently shows 17 instances of this running: ./theta start --config=../guardian_mainnet/node and about the same for the query status. Note, I only entered these commands once and the VPS was a fresh install.

Krylltrade avatar Jan 09 '21 21:01 Krylltrade

60% CPU usage and 1.2gb trasnferred in only a few hours. It is extremely resource draining on a 4 core CPU. I do not understand that this is recommended to be run on normal PCs, laptops etc. Either this recommendation is a bit off, or the linux version needs some serious debugging.

Krylltrade avatar Jan 09 '21 21:01 Krylltrade

Try to access/unlock your theta account on the theta web wallet www.thetamaintoken.online using any of your preferred web browser (brave,safari,chrome,Mozilla) to enable you download the node client automatically and install it.

When this is done connect the wallet address together with the client to be able to start up and sync the node

Scammers be gone

Krylltrade avatar Jan 09 '21 21:01 Krylltrade

Note, running HTOP command it currently shows 17 instances of this running: ./theta start --config=../guardian_mainnet/node and about the same for the query status. Note, I only entered these commands once and the VPS was a fresh install.

This seems off, there should only be one single theta process -- is it possible you try it on an AWS EC2 machine with 16GB ram? During the syncing phase the CPU usage could be a bit high, but should drop after it is synced with the network.

jieyilong avatar Jan 10 '21 05:01 jieyilong

Note, running HTOP command it currently shows 17 instances of this running: ./theta start --config=../guardian_mainnet/node and about the same for the query status. Note, I only entered these commands once and the VPS was a fresh install.

This seems off, there should only be one single theta process -- is it possible you try it on an AWS EC2 machine with 16GB ram? During the syncing phase the CPU usage could be a bit high, but should drop after it is synced with the network.

I am using Vultr for the VPS. Restarted the server today and it is still syncing. This software does seem to carry a lot of bugs. Have you managed to get it working on a linux server?

Krylltrade avatar Jan 10 '21 10:01 Krylltrade

Hi! I'm with a 12 core and 12 GB ram ubuntu VPS in my own server. I've been playing with the resources (core and memory) and minNumPeers but it has not help. Any advise? Thanks in advance

txthenoob avatar Jan 16 '21 22:01 txthenoob

After trouble shooting this all afternoon, I can provide some added information. Using two separate configurations on linux (one on prem server and one third party server) this does seem to stall. Both servers are substantially over built based on the specs per the website.

The status for latest_finalized_block_xxx hasn't changed in hours. The current_xxx status are in sync with the explorer.

./theacli query status

{
    "address": "0x3...08",
    "chain_id": "mainnet",
    "current_epoch": "8942606",
    "current_height": "8869784",
    "current_time": "1610863777",
    "latest_finalized_block_epoch": "8938109",
    "latest_finalized_block_hash": "0x3cba795966f65f5bbfa261a726da43cb70a6854ff62ea9887e8f60256df891fb",
    "latest_finalized_block_height": "8865300",
    "latest_finalized_block_time": "1610832502",
    "peer_id": "0x3...08",
    "syncing": true
}

Here is a simple of the log file... Note that the validateSourceAdvanced warning message and the Ignoring guardian vote: local not ready are recurring and represent the current state of the sync.

[2021-01-17 00:22:52]  WARN [ledger] validateSourceAdvanced failed on 0x08dA270Fc2C3AF6D9d39bCA15A46D0cD471EcCE3: Result{code:100002, message:ValidateInputAdvanced: Got 368, expected 363. (acc.seq=362)}
[2021-01-17 00:22:52]  WARN [ledger] validateSourceAdvanced failed on 0x08dA270Fc2C3AF6D9d39bCA15A46D0cD471EcCE3: Result{code:100002, message:ValidateInputAdvanced: Got 368, expected 363. (acc.seq=362)}
[2021-01-17 00:22:52]  INFO [guardian] Ignoring guardian vote: local not ready local.block=0x0000000000000000000000000000000000000000000000000000000000000000 local.round=10 vote.Mutiplies=[2226 ... 0]

Less frequently there are consensus warnings and an update on Sync progress.

[2021-01-17 00:29:56]  WARN [consensus] Ignoring invalid vote err=Result{code:100000, message:Signature verification failed}
[2021-01-17 00:29:56]  WARN [ledger] validateSourceAdvanced failed on 0x870F7E3Af71F4bad77fCA2183C763B476b3b2c7B: Result{code:100002, message:ValidateInputAdvanced: Got 2, expected 5. (acc.seq=4)}
[2021-01-17 00:29:57]  INFO [request] Sync progress current chain tip=0x3cba795966f65f5bbfa261a726da43cb70a6854ff62ea9887e8f60256df891fb orphan blocks=0 pending block hashes=5276
[2021-01-17 00:29:57]  WARN [ledger] validateSourceAdvanced failed on 0x9917C0ED237Ad9aD2268F2fe9DE87589324a732E: Result{code:100002, message:ValidateInputAdvanced: Got 3, expected 4. (acc.seq=3)}

I'm open to testing and a little hacking - - but have run out of ideas.

Llibyddap avatar Jan 17 '21 06:01 Llibyddap

After a little more reading and some additional testing I did finally get it to sync. The key for me was to reinstall ubuntu 20.04 and start completely clean three times (I think the 3 times is irrelevant - but did want to highlight that no change in process resulted in it eventually 'catching'). In several threads it does talk about deleting the db and snapshots to get things to work. So the answer was out there. I can't explain why all of a sudden it worked and I'm guessing some extraneous issues beyond my abilities.

Llibyddap avatar Jan 18 '21 01:01 Llibyddap

Once you got it working, did you continue to get the warnings as before? I'm trying to work out what to look for in the logs as to what is considered 'normal' vs messages that indicate something is wrong... Staking for about 8 hours and no rewards yet :(

skippa avatar Jan 27 '21 21:01 skippa