reference-based approach for cluster-policy does not work for DLT Pipeline workflows

[djfliu-uplift]

trafficstars

Expected Behavior

"policy_id": "cluster-policy://xxx" should be adjusted to "policy_id": "{the_policy_id}" for DLT Pipeline workflows.

Current Behavior

"policy_id": "cluster-policy://xxx" is used in pipeline definition instead of "policy_id": "{the_policy_id}"

Steps to Reproduce (for bugs)

• dbx version >= 0.8.0
• Create dlt pipeline workflow
• Create a cluster and attach existing policy w/ reference-based approach
• Add cluster to pipeline workflow
• dbx deploy

Your Environment

• dbx versions used: 0.8.3, 0.8.7

Context

It looks like we are missing the Pipeline workflow during the instance check in Line 156 at: dbx/dbx/api/adjuster/adjuster.py

Proposed change: if isinstance(parent, (V2dot0Workflow, JobTaskSettings, JobCluster, Pipeline)) and isinstance(element, NewCluster):

[renardeinside]

hi @djfliu-uplift , policies can be applied in DLT using the built-in functionality via configuration field apply_policy_default_values. At the same time it's a valid point that we should also be able to handle DLT pipelines with named props. I'll take a look at that.