clickhouse-backup icon indicating copy to clipboard operation
clickhouse-backup copied to clipboard

Reducing number of partition in backup

Open mayankkandari opened this issue 4 years ago • 3 comments

Hi, There is no document is available ON this and document pointing to this is empty. https://github.com/AlexAkulov/clickhouse-backup/blob/master/Examples.md#How-to-reduce-number-of-partitions. My requirement is either I pass number of partition to backup or I can pass name of partition to be backup but not all partition.

mayankkandari avatar Dec 23 '20 13:12 mayankkandari

Hi, Can someone please help me in resolving this ?

mayankkandari avatar Jan 06 '21 09:01 mayankkandari

Some research is required to answer this question. I SUPPOSE that if you make a backup and manually create a schema with a different partitioning key and restore only the data from the backup, then it may work. Perhaps some improvements of clickhouse-backup will be required. Maybe this can be done with the regular clickhouse queries like

INSERT INTO xxx SELECT * FROM table WHERE column BETWEEN 1 and 10000;

AlexAkulov avatar Jan 06 '21 12:01 AlexAkulov

I think the idea is to only backup the partitions that we are actively written to, for example the ones that are on hot storage. or just one partition at a time rather than the full table.

see for example : https://www.percona.com/doc/percona-xtrabackup/2.3/howtos/recipes_ibkx_partition.html

aadant avatar Oct 04 '21 21:10 aadant

feature already implemented with create --partitions=XXX,YYY try latest 1.6.2

Slach avatar Sep 03 '22 17:09 Slach