erigon icon indicating copy to clipboard operation
erigon copied to clipboard

`erigon snapshots reset` fails on mounted chaindata directory and may break symlink setups

Open lemenkov opened this issue 3 weeks ago • 6 comments

System information

Erigon version: 3.3.0

OS & Version: Fedora 43 (Aarch64)

Commit hash:

Erigon Command (with flags/config): erigon snapshots reset --datadir /var/lib/erigon/

Consensus Layer: N/A

Consensus Layer Command (with flags/config): N/A

Chain/Network: mainnet

Expected behaviour

erigon snapshots reset should remove the contents of the chaindata directory while preserving the directory itself, any mountpoint, or any symlink.

This is important because the [storage optimization documentation](https://docs.erigon.tech/fundamentals/optimizing-storage) recommends using symlinks:

Store datadir on the slow disk

Place the datadir on the slower disk. Then, create symbolic links (using ln -s) to the fast disk for the following sub-folders:

  • chaindata
  • snapshots/domain

This will speed up the execution of E3.

Actual behaviour

The command attempts to remove the chaindata directory itself rather than its contents. This causes two problems:

  1. Fails when chaindata is a mountpoint with error: removing chaindata dir: unlinkat /var/lib/erigon/chaindata: device or resource busy

  2. May silently break symlink setups - if chaindata is a symlink to fast NVMe storage and reset removes the symlink, the operator may not notice. On next startup, Erigon recreates chaindata as a regular directory on the slower base filesystem, causing significant performance degradation.

Steps to reproduce the behaviour

  1. Mount a separate disk at /var/lib/erigon/chaindata (or create a symlink to another location)
  2. Run erigon snapshots reset --datadir /var/lib/erigon/
  3. Observe failure with "device or resource busy" (mountpoint) or silent removal of symlink

Workaround: Manually clean the directory contents instead:

rm -rf /var/lib/erigon/chaindata/*

Backtrace

[root@curve-7 ~]# erigon snapshots reset --datadir /var/lib/erigon/
INFO[12-02|14:08:54.321] logging to file system                   log dir=/var/lib/erigon/logs file prefix=erigon log level=info json=false
INFO[12-02|14:08:54.321] resetting datadir                        path=/var/lib/erigon
INFO[12-02|14:08:54.324] [db] open                                label=chaindata sizeLimit=8TB pageSize=16KB
INFO[12-02|14:08:54.325] read chain name from config              chain=mainnet
INFO[12-02|14:08:54.325] Loading remote snapshot hashes 
INFO[12-02|14:08:55.918] Loaded preverified snapshots hashes      len=5512 chain=mainnet
INFO[12-02|14:08:55.918] Resetting snapshots directory            path=/var/lib/erigon/snapshots
INFO[12-02|14:08:56.617] Files NOT removed from snapshots directory torrents=1422 data=1422
INFO[12-02|14:08:56.617] Files removed from snapshots directory   torrents=90 data=235
INFO[12-02|14:08:56.617] Removing chaindata dir                   path=/var/lib/erigon/chaindata
removing chaindata dir: unlinkat /var/lib/erigon/chaindata: device or resource busy

lemenkov avatar Dec 02 '25 13:12 lemenkov