gpdb icon indicating copy to clipboard operation
gpdb copied to clipboard

Add pg_upgrade flag --check-not-in-place to run checks relevant to logical upgrade

Open kyeap-vmware opened this issue 11 months ago • 1 comments

Add pg_upgrade flag --check-not-in-place. This flag is meant to be used with upgrade using gpbackup.

The following was assumed when evaluating which checks should be run when this flag is used.

  • oid changes should be ok because metadata and data are reloaded.
  • underlying data type changes because metadata and data are reloaded.
  • indexes do not need to be invalidated because they get remade using CREATE INDEX by gpbackup
  • no access to the target cluster

Here is a list of checks and the reasons for why or why not they were chosen when the not in place upgrade checks are run.

Only applicable to in-place upgrade

check_online_expansion          - gpbackup covers this with gpexpand_sensor
composite_data_type_usage       - As long as the data type still exists gpbackup should be able to handle oid changes
check_for_reg_data_type_usage   - As long as the data type still exists gpbackup should be able to handle oid changes
check_for_jsonb_9_4_usage       - gpbackup should be able to handle internal type changes.
check_for_appendonly_materialized_view_with_relfrozenxid   - Gets resolved when view is remade by gpbackup
old_11_check_for_sql_identifier_data_type_usage            - on disk format change should be handled by gpbackup
old_9_6_invalidate_hash_indexes - Indexes get remade when using gpbackup for upgrade. on disk changes are handled by gpbackup.
isn_and_int8_passing_mismatch   - requires new cluster
check_new_cluster_is_empty      - requires new cluster
check_databases_are_compatible  - requires new cluster
check_loadable_libraries        - requires new cluster
check_for_prepared_transactions - requires new cluster. Guard against in flight on-disk changes?
check_for_new_tablespace_dir    - requires new cluster
check_is_install_user           - requires new cluster
check_proper_datallowconn       - requires new cluster. not using pg_dump.
check_orphaned_toastrels        - Only related to orphaned toastrels. could happen due to kill -9. User relations are gone, this data shouldn't be needed.
                                  We can just leave them behind. gpcheckcat looks for them. Should be a requirement to run gpcheckcat.
check_for_tables_with_oids      - Tables would just get remade to correct. gpbackup doesn't dump with oids.

Required checks for logical upgrade

multi_column_list_partition_keys   - Tables with this partition key are not supported in GPDB7
plpython2_dependent_functions      - plpython2 no longer supported on GPDB7.
views_with_removed_operators       - needed
views_with_removed_functions       - needed
views_with_removed_types           - needed
unsupported_sha256_password_hashes - passwords hashed using sha256 is no longer supported in GPDB7.
removed_data_type_usage_abstime    - needed
removed_data_type_usage_reltime    - needed
removed_data_type_usage_tinterval  - needed
check_for_pg_role_prefix           - roles starting with 'pg_' is no longer allowed on GPDB7
unknown_data_type_usage            - needed

***These checks will either get eventually removed or Gpbackup may be able to handle them eventually, but for now they will be included.
check_partition_indexes            - Check if gpbackup is capable of bringing back wonky partition structures and their indexes. May be possible with the new GP7 syntax
array_of_partition_table_types     - gpbackup doesn't seem to bring back these partition structures, but may eventually handle it.
check_covering_aoindex             - Alters applied may result in partition structure that fails oid check for in-place upgrade. gpbackup may need work to handle this.
check_external_partition           - external partitions are now foreign data wrappers.

pipeline: https://dev.ci.gpdb.pivotal.io/teams/main/pipelines/kyeap-check-not-in-place

gpupgrade check PR https://github.com/greenplum-db/gpupgrade/pull/907 pg_upgrade --check-not-in-place PR https://github.com/greenplum-db/gpdb/pull/17236

kyeap-vmware avatar Mar 22 '24 02:03 kyeap-vmware