dbx
dbx copied to clipboard
TypeError: dbx.api.adjuster.mixins.sql_properties.DashboardsList() argument after ** must be a mapping, not list
trafficstars
Expected Behavior
When defining an Alert SQL task through the deployment.yml like this:
environments:
default:
workflows:
- name: "alert-job"
- task_key: "alert_task"
sql_task:
alert:
alert_id: "alert://alert1"
warehouse_id: "warehouse://sql-warehouse"
it should automatically determine the alert_id through the alert name like it does for warehouses and queries etc.
Current Behavior
Currently I get the following error in my azure pipeline:
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /__w/1/s/prj-venv/lib/python3.9/site-packages/dbx/commands/deploy.py:154 in │
│ deploy │
│ │
│ 151 │ │ if pipelines: │
│ 152 │ │ │ dbx_echo("Found DLT pipelines definition, applying them fi │
│ 153 │ │ │ for elements in [pipelines, workflows]: │
│ ❱ 154 │ │ │ │ adjuster.traverse(elements) │
│ 155 │ │ │ │ wf_manager = WorkflowDeploymentManager(api_client, ele │
│ 156 │ │ │ │ wf_manager.apply() │
│ 157 │ │ else: │
│ │
│ /__w/1/s/prj-venv/lib/python3.9/site-packages/dbx/api/adjuster/adjuster.py:1 │
│ 87 in traverse │
│ │
│ 184 │ │ dbx_echo("Starting the traversal process") │
│ 185 │ │ self.property_adjuster.library_traverse(workflows, self.additi │
│ 186 │ │ self.property_adjuster.file_traverse(workflows, self.file_adju │
│ ❱ 187 │ │ self.property_adjuster.property_traverse(workflows) │
│ 188 │ │ self.property_adjuster.cluster_policy_traverse(workflows) │
│ 189 │ │ dbx_echo("Traversal process finished, all provided references │
│ 190 │
│ │
│ /__w/1/s/prj-venv/lib/python3.9/site-packages/dbx/api/adjuster/adjuster.py:1 │
│ 45 in property_traverse │
│ │
│ 142 │ │ │ │ │ self._adjust_dashboard_ref(element, parent, index) │
│ 143 │ │ │ │ │
│ 144 │ │ │ │ elif element.startswith("alert://"): │
│ ❱ 145 │ │ │ │ │ self._adjust_alert_ref(element, parent, index) │
│ 146 │ │
│ 147 │ def cluster_policy_traverse(self, workflows: WorkflowList): │
│ 148 │ │ """ │
│ │
│ /__w/1/s/prj-venv/lib/python3.9/site-packages/dbx/api/adjuster/mixins/sql_pr │
│ operties.py:116 in _adjust_alert_ref │
│ │
│ 113 │ │
│ 114 │ def _adjust_alert_ref(self, element: str, parent: Any, index: Any) │
│ 115 │ │ alert_name = element.replace("alert://", "") │
│ ❱ 116 │ │ _relevant = DashboardsList( │
│ 117 │ │ │ **self.api_client.perform_query("GET", path="/preview/sql/ │
│ 118 │ │ ) │
│ 119 │ │ _id = _relevant.get(alert_name).id │
TypeError: dbx.api.adjuster.mixins.sql_properties.DashboardsList() argument
after ** must be a mapping, not list
Steps to Reproduce (for bugs)
- Create deployment.yml with the following:
environments:
default:
workflows:
- name: "alert-job"
- task_key: "alert_task"
sql_task:
alert:
alert_id: "alert://alert1"
warehouse_id: "warehouse://sql-warehouse"
- Deploy.
Context
Your Environment
- dbx version used: 0.8.18
On a first glance I think the reason for the error might be that in the _adjust_alert_refmethod of the SqlPropertiesAdjusterclass (line 116) in dbx/api/adjuster/mixins/sql_properties.py it should reference the class AlertsList instead of DashboardsList:
.
.
.
class DashboardsList(FlexibleModel, ResultsListGetterMixin):
results: List[DashboardInfo] = []
@property
def object_type(self) -> str:
return "dashboard"
class AlertsList(FlexibleModel, ResultsListGetterMixin):
results: List[AlertInfo] = []
@property
def object_type(self) -> str:
return "alert"
class SqlPropertiesAdjuster(ApiClientMixin, ElementSetterMixin):
# TODO: design of this class is a terrible copy-paste. It must be rewritten.
@functools.cached_property
def _warehouses(self) -> WarehousesList:
return WarehousesList(**self.api_client.perform_query("GET", path="/sql/warehouses/"))
def _adjust_warehouse_ref(self, element: str, parent: Any, index: Any):
_id = self._warehouses.get(element.replace("warehouse://", "")).id
self.set_element_at_parent(_id, parent, index)
def _adjust_query_ref(self, element: str, parent: Any, index: Any):
query_name = element.replace("query://", "")
_relevant = QueriesList(
**self.api_client.perform_query("GET", path="/preview/sql/queries", data={"q": query_name})
)
_id = _relevant.get(query_name).id
self.set_element_at_parent(_id, parent, index)
def _adjust_dashboard_ref(self, element: str, parent: Any, index: Any):
dashboard_name = element.replace("dashboard://", "")
_relevant = DashboardsList(
**self.api_client.perform_query("GET", path="/preview/sql/dashboards", data={"q": dashboard_name})
)
_id = _relevant.get(dashboard_name).id
self.set_element_at_parent(_id, parent, index)
def _adjust_alert_ref(self, element: str, parent: Any, index: Any):
alert_name = element.replace("alert://", "")
_relevant = AlertsList(
**self.api_client.perform_query("GET", path="/preview/sql/alerts", data={"q": alert_name})
)
_id = _relevant.get(alert_name).id
self.set_element_at_parent(_id, parent, index)
However, I haven't confirmed this. (When I provide the explicit alert_id the deployment goes through without any problems btw.)