citus
citus copied to clipboard
Use TRUNCATE + COPY FREEZE when copying a shard
In src/backend/distributed/operations/worker_shard_copy.c
file contains our main COPY logic for for shard moves and shard splits. After talking with @DimCitus I realized that we could use the FREEZE
option of COPY
to reduce the the need for heavy vacuuming after the copy is done. To benefit from the freeze option it's required to truncate the target table in the same transaction as the copy. This is fine for all our (current) use cases, because the target shard has just been created and is thus empty.
What I think is needed to achieve this:
- Add the
FREEZE
option to the COPY command that we generate inConstructShardCopyStatement
- Add the
FREEZE
option to the list of options created inLocalCopyToShard
- Start a transaction in
ConnectToRemoteAndStartCopy
- Truncate the table in
ConnectToRemoteAndStartCopy
(before starting the COPY) - End the transaction in
ShardCopyDestReceiverShutdown
Apart from the actual implementation this needs tests to see that indeed vacuum is not necessary on the new table after a shard move.
I wonder whether statistics gathering would be affected. It might anyway be good to do an ANALYZE as part of a move to get statistics.