clickhouse-backup icon indicating copy to clipboard operation
clickhouse-backup copied to clipboard

Test / add limited (?) support for s3 disks in ClickHouse

Open filimonov opened this issue 3 years ago • 3 comments

We need to be sure we do UNFREEZE to release the files which were freezed.

filimonov avatar Apr 08 '22 16:04 filimonov

could you explain what exactly you mean?

Slach avatar Apr 08 '22 16:04 Slach

I didn't test that (need to be checked) but it looks like that:

If we have s3 disk configured in clickhouse, then freeze command create a 'hardlinks' in shadow dir (both in local FS and in S3) which increment the refcount to the target object.

Even if you remove the file from the local filesystem (or move it to backup subfolder), the 'hardlink' in the virtual filesystem (in s3) stays in place. And it don't allow the data files to be deleted after the part go away (because refcount never goes back to zero).

There is 'unfreeze' command introduced for that (?)

filimonov avatar Apr 08 '22 16:04 filimonov

got it

Slach avatar Apr 08 '22 17:04 Slach

currently when we setup

clickhouse:
  use_embedded_backup_restore: true

s3 disks should be also supported

Slach avatar Mar 30 '23 10:03 Slach