schemachange icon indicating copy to clipboard operation
schemachange copied to clipboard

Add Always Execute First Functionality

Open CaptCorpMURICA opened this issue 1 year ago • 6 comments

Description:

Add the functionality to execute specific scripts first on every execution of schemachange using the naming convention F__DESCRIPTION.sql.

Benefit:

For use in CI/CD process:

  1. Create clone of production environment to create the QA environment.
  2. Apply permissions required for the QA environment.
  3. Execute rest of schemachange workflow to apply deployment script to QA environment. If this fails, then the deployment would fail a production execution.

By tagging scripts to be executed first, schemachange can create the cloned environment and apply the appropriate permissions before it attempts to deploy any changes.

Best Case Enhancement:

The functionality would be improved if an optional condition could be applied so it is only executed against QA and not to PROD, since it would not be needed against that environment. For instance, if the naming convention was F__QA_DESCRIPTION.sql, then it would only run against the QA environment and F__STG_DESCRIPTION.sql would only run against the staging environment. This enhancement is not part of the MVP of this feature but would require schemachange to analyze the database to match a REGEX for which files to run or omit. It would not work unless specific naming conventions were utilized in Snowflake and is out of scope for this initial enhancement.

This could also be implemented with a command line argument like, -af, --always-first or in the YAML file of always-first: TRUE. This would enable the YAML for the Dev to QA pipeline to leverage these files but would not include it in the QA to Prod pipeline.

Currently, we use SnowSQL to execute stored procedures for cloning the environment and applying the grants. Once that completes, then the schemachange execution occurs. It would be ideal to merge this functionality with schemachange to keep the entire process contained into a single tool and reduce additional technical debt.

CaptCorpMURICA avatar Mar 02 '23 20:03 CaptCorpMURICA