programmable configuration API

[renardeinside]

trafficstars

[copdips]

Hello @renardeinside

I have an idea that uses pure Python to generate the deployement definition. User should provide a python function that generates a dict as workflow definition, and dbx just handles the dict in memory without any deployment file.

The benefit of using pure Python is that there's no more jinja or var files, and we can leverage Python to reduce at maximum and also lint the deployment code.

1. user writes a python function which should output a dict object for the deployement definition:

""" file path: conf/deployment.py
"""

def get_deployment_dict(env: str) -> dict:
    return {"environments": {env: {"workflows": []}}}

2. user run dbx deploy with this python function, and its params:

dbx deploy \
     --deployment-function conf.deployment:get_deployment_dict \
     --deployment-function-params '{"env": "dev"}'

3. dbx deploy loads the dict

import importlib
import json


deploy_module_path, deploy_function_name = deployment_function.split(":")
deploy_module = importlib.import_module(deploy_module_path)
deployment_function_params_dict = json.loads(deployment_function_params)
deploy_dict = getattr(deploy_module, deploy_function_name)(**deployment_function_params_dict)
# from now on, dbx has a dict object of the definition, then enjoy.