cloudberry icon indicating copy to clipboard operation
cloudberry copied to clipboard

Replace the hacky way stats pg_appendonly.segfilecount

Open avamingli opened this issue 1 year ago • 1 comments

Cloudberry Database version

No response

What happened

This is added by me when implementing AO/AOCS's parallel scan feature.

segfilecount is used to parallel workers for AO/AOCS. It's updated by VACCUM, ANALYZE, TRUNCATE command and auto vacuum process. I use VPgClassStats.relallvisible to represent segment file count of AO/AOCS as we are hurry for that feature, and I could save a MessageType of libpq and etc.

It works well because AO/AOCS does not currently have an equivalent to Heap's 'all visible pages', relallvisible is always set to 0 in pg_class for AO/AOCS tabes. But QE use this field in libpq to represent AO/AOCS's total file segment number when vacuum ao tables. See more comments in below files.

typedef struct VPgClassStats
{
	Oid			relid;
	BlockNumber rel_pages;
	double		rel_tuples;
	BlockNumber relallvisible;
} VPgClassStats;

The approach is very hacky and not good. We should reconsider the implementation for this including VACUUM, ANALYZE, TRUNCATE and auto vacuum process.

vacuum_ao.c

 /* AO/AOCO does not currently have an equivalent to
							  Heap's 'all visible pages', use this field to represent
							  AO/AOCO's total segment file count */

vacuum.h

 * relallvisible
 * AO/AOCO does not currently have an equivalent to Heap's 'all visible pages',
 * relallvisible is always set to 0 in pg_class for AO/AOCO tabes. But QE use
 * this field in libpq to represent AO/AOCO's total file segment number when
 * vacuum ao tables.
 * Remember to reset relallvisible to 0 after qd get file segment number and
 * before updating pg_class.
 * See vac_update_relstats_from_list in vacuum.c and
 * vac_update_relstats in vacuum_ao.c

What you think should happen instead

No response

How to reproduce

No need.

Operating System

Ubuntu.

Anything else

No response

Are you willing to submit PR?

  • [ ] Yes, I am willing to submit a PR!

Code of Conduct

avamingli avatar Jul 26 '23 06:07 avamingli