raspiblitz icon indicating copy to clipboard operation
raspiblitz copied to clipboard
verified publisher icon rootzoll

Sync process always cancels.

Open wilson-ok opened this issue 1 year ago • 3 comments

Hello, I need some help. I have been trying to re-sync a Raspiblitz for a week now. I always get stuck at about 8% and keep getting the following message:

Bildschirmfoto 2024-06-22 um 11 37 26

I have followed the hardware recommendation exactly and have already reflashed the SD. I have already tried to reproduce similar errors. Unfortunately, I don't know what to do and it's not really fun at the moment.

Best regards

Here is the debug report:

  • RASPIBLITZ DEBUG LOGS

blitzversion: 1.11.0 chainnetwork: bitcoin / test 10:39:05 up 3:50, 4 users, load average: 1.25, 1.21, 1.12

*** FAILED SERVICES *** list any servcies with problems: sudo systemctl list-units --failed UNIT LOAD ACTIVE SUB DESCRIPTION ● bitcoind.service loaded failed failed Bitcoin daemon on mainnet ● smartmontools.service loaded failed failed Self Monitoring and Reporting Technology (SMART) Daemon

LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 2 loaded units listed.

*** SETUPPHASE / BOOTSTRAP *** see logs: cat /home/admin/raspiblitz.log setupPhase--> done state--> ready

*** BACKGROUNDSERVICE *** to monitor Background service call: sudo journalctl -f -u background

*** BLOCKCHAIN (MAINNET) SYSTEMD STATUS *** × bitcoind.service - Bitcoin daemon on mainnet Loaded: loaded (/etc/systemd/system/bitcoind.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Sat 2024-06-22 07:07:55 BST; 3h 31min ago Duration: 17min 13.105s Process: 94926 ExecStartPre=/home/admin/config.scripts/bitcoin.check.sh prestart mainnet (code=exited, status=0/SUCCESS) Process: 94940 ExecStart=/usr/local/bin/bitcoind -daemonwait -conf=/mnt/hdd/bitcoin/bitcoin.conf -datadir=/mnt/hdd/bitcoin (code=exited, status=1/FAILURE) CPU: 199ms

Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Failed with result 'exit-code'. Jun 22 07:07:55 raspberrypi systemd[1]: Failed to start bitcoind.service - Bitcoin daemon on mainnet.

*** LAST BLOCKCHAIN (MAINNET) ERROR LOGS *** sudo journalctl -u bitcoind -b --no-pager -n20 Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Scheduled restart job, restart counter is at 5. Jun 22 07:07:55 raspberrypi systemd[1]: Stopped bitcoind.service - Bitcoin daemon on mainnet. Jun 22 07:07:55 raspberrypi systemd[1]: Starting bitcoind.service - Bitcoin daemon on mainnet... Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94927]: /bin/chgrp: changing group of '/mnt/hdd/bitcoin': Read-only file system Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94928]: sed: can't read /mnt/hdd/bitcoin/bitcoin.conf: Input/output error Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94929]: sed: can't read /mnt/hdd/bitcoin/bitcoin.conf: Input/output error Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94930]: grep: /mnt/hdd/bitcoin/bitcoin.conf: Input/output error Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94931]: sed: can't read /mnt/hdd/bitcoin/bitcoin.conf: Input/output error Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94932]: touch: cannot touch '/mnt/hdd/bitcoin/debug.log': Read-only file system Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94933]: chown: changing ownership of '/mnt/hdd/bitcoin/debug.log': Read-only file system Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94934]: chmod: changing permissions of '/mnt/hdd/bitcoin/debug.log': Read-only file system Jun 22 07:07:55 raspberrypi bitcoind[94940]: Error: Error reading configuration file: specified config file "/mnt/hdd/bitcoin/bitcoin.conf" could not be opened. Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Control process exited, code=exited, status=1/FAILURE Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Failed with result 'exit-code'. Jun 22 07:07:55 raspberrypi systemd[1]: Failed to start bitcoind.service - Bitcoin daemon on mainnet. Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Scheduled restart job, restart counter is at 6. Jun 22 07:07:55 raspberrypi systemd[1]: Stopped bitcoind.service - Bitcoin daemon on mainnet. Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Start request repeated too quickly. Jun 22 07:07:55 raspberrypi systemd[1]: bitcoind.service: Failed with result 'exit-code'. Jun 22 07:07:55 raspberrypi systemd[1]: Failed to start bitcoind.service - Bitcoin daemon on mainnet.

*** LAST BLOCKCHAIN (MAINNET) INFO LOGS *** sudo tail -n 50 /mnt/hdd/bitcoin/debug.log

*** LND (MAINNET) SYSTEMD STATUS ***

  • OFF by config -

*** CORE LIGHTNING (MAINNET) SYSTEMD STATUS ***

  • not activated -

*** BLOCKCHAIN (TESTNET) SYSTEMD STATUS ***

  • OFF by config - *** LND (TESTNET) SYSTEMD STATUS ***
  • OFF by config -

*** CORE LIGHTNING (TESTNET) SYSTEMD STATUS ***

  • not activated -

*** BLOCKCHAIN (SIGNET) SYSTEMD STATUS ***

  • OFF by config - *** LND (SIGNET) SYSTEMD STATUS ***
  • OFF by config -

*** CORE LIGHTNING (SIGNET) SYSTEMD STATUS ***

  • not activated -

*** NGINX SYSTEMD STATUS *** ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; preset: enabled) Drop-In: /etc/systemd/system/nginx.service.d └─raspiblitz.conf Active: active (running) since Sat 2024-06-22 06:46:15 BST; 3h 52min ago Docs: man:nginx(8) Main PID: 1854 (nginx) Tasks: 5 (limit: 4454) CPU: 433ms CGroup: /system.slice/nginx.service ├─1854 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;" ├─1855 "nginx: worker process" ├─1856 "nginx: worker process" ├─1857 "nginx: worker process" └─1858 "nginx: worker process"

Jun 22 06:46:13 raspberrypi systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Jun 22 06:46:15 raspberrypi systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server.

*** LAST NGINX LOGS *** sudo journalctl -u nginx -b --no-pager -n20 Jun 22 06:46:13 raspberrypi systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Jun 22 06:46:15 raspberrypi systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server. --> CHECK CONFIG: sudo nginx -t 2024/06/22 10:39:05 [emerg] 1251380#1251380: cannot load certificate "/mnt/hdd/app-data/nginx/tls.cert": BIO_new_file() failed (SSL: error:80000005:system library::Input/output error:calling fopen(/mnt/hdd/app-data/nginx/tls.cert, r) error:10080002:BIO routines::system lib) nginx: configuration file /etc/nginx/nginx.conf test failed

*** BLITZAPI STATUS *** installed=1 repo='https://github.com/fusion44/blitz_api.git' branch='blitz-v1.11' commit='b08e53b599696edeeeaef1081caad6ad78900b48' ● blitzapi.service - BlitzBackendAPI Loaded: loaded (/etc/systemd/system/blitzapi.service; enabled; preset: enabled) Active: active (running) since Sat 2024-06-22 06:46:09 BST; 3h 52min ago Main PID: 967 (python) Tasks: 14 (limit: 4454) CPU: 3min 5.700s CGroup: /system.slice/blitzapi.service └─967 /home/blitzapi/blitz_api/venv/bin/python -m uvicorn app.main:app --port 11111 --host=... --root-path /api

Jun 22 10:39:05 raspberrypi python[967]: 2024-06-22 10:39:05 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:39:05 raspberrypi python[967]: 2024-06-22 10:39:05 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB.

*** LAST BLITZAPI LOGS *** sudo journalctl -u blitzapi -b --no-pager -n20 Jun 22 10:38:53 raspberrypi python[967]: 2024-06-22 10:38:53 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:38:53 raspberrypi python[967]: 2024-06-22 10:38:53 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB. Jun 22 10:38:55 raspberrypi python[967]: 2024-06-22 10:38:55 | ❌ | service.py:210 | Aiohttp client connection error: Cannot connect to host ...:8332 ssl:default [Connect call failed ('127.0.0.1', 8332)] Jun 22 10:38:55 raspberrypi python[967]: 2024-06-22 10:38:55 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:38:55 raspberrypi python[967]: 2024-06-22 10:38:55 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB. Jun 22 10:38:57 raspberrypi python[967]: 2024-06-22 10:38:57 | ❌ | service.py:210 | Aiohttp client connection error: Cannot connect to host ...:8332 ssl:default [Connect call failed ('127.0.0.1', 8332)] Jun 22 10:38:57 raspberrypi python[967]: 2024-06-22 10:38:57 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:38:57 raspberrypi python[967]: 2024-06-22 10:38:57 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB. Jun 22 10:38:59 raspberrypi python[967]: 2024-06-22 10:38:59 | ❌ | service.py:210 | Aiohttp client connection error: Cannot connect to host ...:8332 ssl:default [Connect call failed ('127.0.0.1', 8332)] Jun 22 10:38:59 raspberrypi python[967]: 2024-06-22 10:38:59 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:38:59 raspberrypi python[967]: 2024-06-22 10:38:59 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB. Jun 22 10:39:01 raspberrypi python[967]: 2024-06-22 10:39:01 | ❌ | service.py:210 | Aiohttp client connection error: Cannot connect to host ...:8332 ssl:default [Connect call failed ('127.0.0.1', 8332)] Jun 22 10:39:01 raspberrypi python[967]: 2024-06-22 10:39:01 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:39:01 raspberrypi python[967]: 2024-06-22 10:39:01 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB. Jun 22 10:39:03 raspberrypi python[967]: 2024-06-22 10:39:03 | ❌ | service.py:210 | Aiohttp client connection error: Cannot connect to host ...:8332 ssl:default [Connect call failed ('127.0.0.1', 8332)] Jun 22 10:39:03 raspberrypi python[967]: 2024-06-22 10:39:03 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:39:03 raspberrypi python[967]: 2024-06-22 10:39:03 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB. Jun 22 10:39:05 raspberrypi python[967]: 2024-06-22 10:39:05 | ❌ | service.py:210 | Aiohttp client connection error: Cannot connect to host ...:8332 ssl:default [Connect call failed ('127.0.0.1', 8332)] Jun 22 10:39:05 raspberrypi python[967]: 2024-06-22 10:39:05 | ℹ️ | utils.py:67 | Key 'tor_web_addr' not found in Redis DB. Jun 22 10:39:05 raspberrypi python[967]: 2024-06-22 10:39:05 | ⚠️ | utils.py:69 | Key 'hdd_capacity_bytes' not found in Redis DB.

*** BLITZ WebUI STATUS *** installed=1 repo='https://github.com/raspiblitz/raspiblitz-web.git' branch='release/v1.11' commit='3f7d7a48e7a32ff8fb1b3f7333941687b62099f1'

  • TOUCHSCREEN is OFF by config
  • Loop is OFF by config
  • LND-RTL is OFF by config
  • CL-RTL is OFF by config
  • Electrum Rust Server is OFF by config
  • LIT is OFF by config
  • LNDg is OFF by config
  • BTCPayServer is OFF by config
  • BTC-RPC-Explorer is OFF by config
  • LNbits is OFF by config
  • Thunderhub is OFF by config
  • SPECTER is OFF by config
  • SPHINX is OFF by config
  • FINTS is OFF by config

*** MOUNTED DRIVES *** df -T -h Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 1.9G 0 1.9G 0% /dev tmpfs tmpfs 380M 1.1M 379M 1% /run /dev/mmcblk0p2 ext4 58G 27G 29G 49% / tmpfs tmpfs 1.9G 8.0K 1.9G 1% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 32M 12K 32M 1% /var/cache/raspiblitz /dev/mmcblk0p1 vfat 510M 64M 447M 13% /boot/firmware /dev/sda1 ext4 916G 78G 793G 9% /mnt/hdd tmpfs tmpfs 380M 32K 380M 1% /run/user/1000 tmpfs tmpfs 380M 32K 380M 1% /run/user/1001

*** SD CARD HOMES *** sudo du -ahd1 /home 2.6G /home/btcpay 325M /home/lnbits 2.2G /home/mempool 2.4G /home/electrs 496M /home/blitzapi 312M /home/btcrpcexplorer 560M /home/jam 623M /home/joinmarket 810M /home/bitcoin 461M /home/admin 55M /home/pi 436M /home/rtl 1.5G /home/thunderhub 13G /home

*** LOGFILES *** Archived and active journals take up 264.0M in the file system. 582M /var/log

*** DATADRIVE ***

raspi_bootdir(/boot/firmware)

RASPIBLITZ DATA DRIVE Status

BASICS

isMounted=1 isBTRFS=0 hddRaspiData=0 hddRaspiVersion='' isSMART=0 isSSD=1 datadisk='sda' datapartition='sda1' hddCandidate='sda' hddPartitionCandidate='sda1' hddTemperature= hddBlocksBitcoin=0 hddBytes= hddGigaBytes= hddTBSize='<1TB' hddUsedInfo='<1TB 491139%' error='unkown command' smartctl 7.3 2022-02-28 r5338 [aarch64-linux-6.6.20+rpt-rpi-v8] (local build) Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

Smartctl open device: /dev/sda failed: No such device

*** NETWORK *** localip=... localiprange=.../24 dhcp=1 network_device=eth0

*** ZRAM *** downloaded=1 serviceLoaded=1 serviceActive=1

*** HARDWARE TEST RESULTS *** throttled=0x0 UndervoltageReports in Logs: 0

*** SYSTEM CACHE STATUS *** system_ups_status="OFF" system_ups_battery="" system_count_start_blockchain="6" system_ram_gb="4" system_up="13815" system_count_start_lightning="1" system_temp_celsius="49" system_ram_mb="3792" system_temp_fahrenheit="120" system_init_time="1719035172" system_count_undervoltage="0" system_ram_available_mb="3409" system_count_longscan="1" system_vm_vagrant="0" system_board="rp4" system_cpu_load=" 1.25, 1.21, 1.12" ln_default_running="0" ln_default_version="0.17.3-beta" ln_default_wallet_onchain_balance=**** ln_default_peers="0" ln_default_channels_total="0" ln_default_channels_inactive="0" ln_default_recovery_mode="0" ln_default_sync_initial_done="0" ln_default_error_short="" ln_default_wallet_channels_pending="0" ln_default_wallet_channels_balance=**** ln_default_activated="1" ln_default_online="0" ln_default_locked="0" ln_default_recovery_done="0" ln_default_alias=**** ln_default_fees_total="0" ln_default_channels_active="0" ln_default_error_full="" ln_default_ready="0" ln_default_sync_progress="44.56" ln_default_tor="0" ln_default_sync_chain="0" ln_default_channels_pending="0" ln_default_wallet_onchain_pending="0" btc_default_running="0" btc_default_sync_percentage="9.13" btc_default_ready="0" btc_default_error_full="" btc_default_error_short="" btc_default_blocks_behind="463996" btc_default_sync_initial_done="0" btc_default_synced="0" btc_default_blocks_headers="848944" btc_default_version="v26.0.0" btc_default_mempool_transactions="0" btc_default_online="0" btc_default_sync_progress="0.09126662844208179" btc_default_blocks_data_kb="71547988" btc_default_sync_initialblockdownload="1" btc_default_peers="4" btc_default_port="8333" btc_default_activated="1" btc_default_blocks_verified="384948" *** POSSIBLE ERROR REPORTS ***

wilson-ok avatar Jun 22 '24 09:06 wilson-ok

It seems that your filesystem of the SSD slips into a "read-only" state:

Jun 22 07:07:55 raspberrypi systemd[1]: Starting bitcoind.service - Bitcoin daemon on mainnet...
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94927]: /bin/chgrp: changing group of '/mnt/hdd/bitcoin': Read-only file system
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94928]: sed: can't read /mnt/hdd/bitcoin/bitcoin.conf: Input/output error
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94929]: sed: can't read /mnt/hdd/bitcoin/bitcoin.conf: Input/output error
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94930]: grep: /mnt/hdd/bitcoin/bitcoin.conf: Input/output error
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94931]: sed: can't read /mnt/hdd/bitcoin/bitcoin.conf: Input/output error
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94932]: touch: cannot touch '/mnt/hdd/bitcoin/debug.log': Read-only file system
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94933]: chown: changing ownership of '/mnt/hdd/bitcoin/debug.log': Read-only file system
Jun 22 07:07:55 raspberrypi bitcoin.check.sh[94934]: chmod: changing permissions of '/mnt/hdd/bitcoin/debug.log': Read-only file system
Jun 22 07:07:55 raspberrypi bitcoind[94940]: Error: Error reading configuration file: specified config file "/mnt/hdd/bitcoin/bitcoin.conf" could not be opened.

Why this is - not clear ..

"Smartctl open device: /dev/sda failed: No such device" but df says its mounted

*** MOUNTED DRIVES ***
df -T -h
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda1 ext4 916G 78G 793G 9% /mnt/hdd
...

@openoms have you seen such error in the past?

rootzoll avatar Jun 27 '24 20:06 rootzoll

Yes, it is a better form of SSD failure.

Please send another: debug -l report link after restart or to see how long the SSD was used. If the disk is new it's likely a power failure which could be solves with using an X825 SATA adapter board with the barrel connector. That can drive the 2TB disks as well.

openoms avatar Jun 28 '24 04:06 openoms

Hello

The error seems to have been found. I have replaced another SSD housing and since then no more crashes. However, I had used the enclosure from the hardware recommendation. I had the problems with this one.

Thanks for the help.

wilson-ok avatar Jun 28 '24 15:06 wilson-ok