data-api-builder icon indicating copy to clipboard operation
data-api-builder copied to clipboard

Consolidate `dbtype-commands.txt` files for easier maintainability

Open severussundar opened this issue 3 years ago • 4 comments

As a next step to #471, move the commands that are common to all database types to a common file.

Benefits in performing this:

  1. This ensures that changes have to be made at only one place
  2. Latest updates to config files will be reflected across all SQL database types.

severussundar avatar Oct 27 '22 04:10 severussundar

Moving this to Jan2023 - since this is cleanup and we have other higher priority items.

Aniruddh25 avatar Nov 10 '22 05:11 Aniruddh25

Moving to March2023 as this is a refactoring effort and there are higher priority items.

severussundar avatar Jan 06 '23 05:01 severussundar

We have scripts for each db type : mssql, mysql, pgsql, dwsql.

Completing this task requires:

  • all automation scripts in https://github.com/Azure/data-api-builder/tree/main/config-generators should create a db type agnostic file, dab-config.json, that will us to generalize the automation scripts.
add Broker --config "dab-config.MySql.json" --source brokers --permissions "anonymous:read"

The above dab cli execution step doesn't have to be tightly coupled to a database type. The config-generator.sh/ps1 files no longer need to set specific db file names like:

#Generates the config files for the selected database types.
foreach($databaseType in $databaseTypes){
    if($databaseType -eq "mssql"){
        $commandFile = "mssql-commands.txt";
        $configFile = "dab-config.MsSql.json";
    }
    elseif($databaseType -eq "mysql"){
        $commandFile = "mysql-commands.txt";
        $configFile = "dab-config.MySql.json";
    }
    elseif($databaseType -eq "postgresql"){
        $commandFile = "postgresql-commands.txt";
        $configFile = "dab-config.PostgreSql.json";
    }
    elseif($databaseType -eq "dwsql"){
        $commandFile = "dwsql-commands.txt";
        $configFile = "dab-config.DwSql.json";
    }
    else{
        $commandFile = "cosmosdb_nosql-commands.txt";
        $configFile = "dab-config.CosmosDb_NoSql.json";
    }

csproj post-build step in our pipelines only ever create one of these files to use, never all 4. So file name overlap is not expected (which is why i suspect we had code like the above in the first place).

We would only really need two lines like the following:

        # <code here to run dab init --database-type {mssql, dwsql, pgsql, mysql}>  this is the only place db type is needed. the dab init step also sets our only db specific config settings like set session context (mssql) and graphqlschema file (cosmosdb)
        # generic file names
        $commandFile = "dab-setup-commands.txt";
        $configFile = "dab-config.json";

seantleonard avatar May 17 '24 00:05 seantleonard

Setting higher priority because this will help with local dev experience. I've dealt with how tediuous it is to troubleshoot/remember to add config automation steps to ALL automation scripts.

seantleonard avatar May 17 '24 00:05 seantleonard