Incorrect Compile Code from DBT Power User

[ashutran]

Expected behavior

While using the Complied DBT preview feature result is not as expected:

1. In my model, I'm using a macro for the incremental logic, which contains various if-else statements to determine the compiled code. However, it seems that part of the logic is being skipped, and I'm not sure why this is happening.

2. I compared the compiled code generated by DBT with the code generated by the plugin, and I noticed that several pieces of logic that should be present in the compiled code are missing.

3. In my model, after implementing the incremental logic, I use another macro to apply the casting logic to ensure each column is cast to its appropriate type. However, this step is failing.

4. While executing the query, it's failing with the following error. My DBT architecture places all models under a folder called CDW. However, using dbt deps, we're downloading the project dependencies into a core folder, as specified in the packages.yml file.

{ "code": -1, "message": "Database Error\n Syntax error: SELECT list must not be empty at [305:1]", "data": ""Error: Database Error\n Syntax error: SELECT list must not be empty at [305:1]\n\tat DBTCoreProjectIntegration_1. (c:\\Users\\.vscode\\extensions\\innoverio.vscode-dbt-power-user-0.39.12\\dist\\extension.js:18398:127)\n\tat Generator.throw ()\n\tat rejected (c:\\Users\\.vscode\\extensions\\innoverio.vscode-dbt-power-user-0.39.12\\dist\\extension.js:25808:28)"" }

Actual behavior

After compiling the code

1. It should read the macro logic and based on that it should return the correct code.
2. Should apply the casting logic as mentioned the model using the specific Macro.
3. Query Result should return the value

Steps To Reproduce

Using poetry shell to activate the poetry virtual env Using python version 3.11.6 dbt-core 1.6.1 dbt-bigquery 1.6.4

Create any model having Macro for incremental logic using if-else where it'll check if the table exist then if condition else another condition

Log output/Screenshots

No response

Operating System

Windows 10

dbt version

1.6.1

dbt Adapter

dbt-bigquery 1.6.4

dbt Power User version

v0.39.12

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[sweep-ai-deprecated[bot]]

🚀 Here's the PR! #1159

💎 Sweep Pro: You have unlimited Sweep issues

Actions

• [ ] ↻ Restart Sweep

Step 1: 🔎 Searching

Here are the code search results. I'm now analyzing these search results to write the PR.

Relevant files (click to expand). Mentioned files will always appear here.

https://github.com/AltimateAI/vscode-dbt-power-user/blob/76cd20da0218230f0d18d18ac79986087558215e/dbt_core_integration.py#L1-L825

https://github.com/AltimateAI/vscode-dbt-power-user/blob/76cd20da0218230f0d18d18ac79986087558215e/src/dbt_client/dbtCoreIntegration.ts#L1-L1089

https://github.com/AltimateAI/vscode-dbt-power-user/blob/76cd20da0218230f0d18d18ac79986087558215e/altimate_packages/altimate/utils.py#L1-L350

https://github.com/AltimateAI/vscode-dbt-power-user/blob/76cd20da0218230f0d18d18ac79986087558215e/altimate_packages/sqlglot/generator.py#L1-L2731

Step 2: ⌨️ Coding

dbt_core_integration.py

Add debugging statements to the `_compile_node` method to log the progress of the incremental logic macro execution.

--- 
+++ 
@@ -13,6 +13,21 @@
             else:
                 # this is essentially a convenient wrapper to adapter.get_compiler
                 compiled_node = self.sql_compiler.compile(self.dbt)
+
+            # Add debugging statements
+            self.dbtTerminal.debug(
+                "_compile_node",
+                f"Compiling node: {node.unique_id}",
+            )
+            self.dbtTerminal.debug(
+                "_compile_node",
+                f"Raw SQL: {getattr(compiled_node, RAW_CODE)}",
+            )
+            self.dbtTerminal.debug(
+                "_compile_node",
+                f"Compiled SQL: {getattr(compiled_node, COMPILED_CODE)}",
+            )
+
             return DbtAdapterCompilationResult(
                 getattr(compiled_node, RAW_CODE),
                 getattr(compiled_node, COMPILED_CODE),

dbt_core_integration.py

Verify that the `sql_compiler` is properly set up and configured to handle macros with if-else statements.

--- 
+++ 
@@ -7,4 +7,13 @@
             self._sql_compiler = SqlCompileRunner(
                 self.config, self.adapter, node=None, node_index=1, num_nodes=1
             )
+
+            # Verify sql_compiler configuration
+            assert self._sql_compiler.config.has_option("compile", "handle_macros"), (
+                "sql_compiler configuration missing 'handle_macros' option"
+            )
+            assert self._sql_compiler.config.get("compile", "handle_macros"), (
+                "sql_compiler not configured to handle macros"
+            )
+
         return self._sql_compiler

dbt_core_integration.py

Ensure that the `compile` method of the `sql_compiler` is being called with the correct arguments, including the dbt manifest.

--- 
+++ 
@@ -12,7 +12,7 @@
                 )
             else:
                 # this is essentially a convenient wrapper to adapter.get_compiler
-                compiled_node = self.sql_compiler.compile(self.dbt)
+                compiled_node = self.sql_compiler.compile(manifest=self.dbt)
             return DbtAdapterCompilationResult(
                 getattr(compiled_node, RAW_CODE),
                 getattr(compiled_node, COMPILED_CODE),

dbt_core_integration.py

Add logging statements in the `_compile_node` method to capture the compiled code generated by dbt and the plugin.

--- 
+++ 
@@ -13,10 +13,25 @@
             else:
                 # this is essentially a convenient wrapper to adapter.get_compiler
                 compiled_node = self.sql_compiler.compile(self.dbt)
-            return DbtAdapterCompilationResult(
+
+            # Log compiled code
+            self.dbtTerminal.debug(
+                "_compile_node",
+                f"DBT compiled code: {getattr(compiled_node, COMPILED_CODE)}",
+            )
+
+            result = DbtAdapterCompilationResult(
                 getattr(compiled_node, RAW_CODE),
                 getattr(compiled_node, COMPILED_CODE),
                 compiled_node,
             )
+
+            # Log plugin compiled code
+            self.dbtTerminal.debug(
+                "_compile_node",
+                f"Plugin compiled code: {result.compiled_sql}",
+            )
+
+            return result
         except Exception as e:
             raise Exception(str(e))

Step 3: 🔄️ Validating

Your changes have been successfully made to the branch sweep/incorrect_compile_code_from_dbt_power_us. I have validated these changes using a syntax checker and a linter.

---

[!TIP] To recreate the pull request, edit the issue title or description.

This is an automated message generated by Sweep AI.

[anandgupta42]

@mdesmet Can you have a look?

[mdesmet]

@ashutran : Can you provide us with a sample code that reproduces the issue(s)? I think we don't have the full picture of what you are trying to achieve.

[ZachMassia]

I think I may have run into this just now. My google searching simply leads me back here.

Here is a simple way to test missing conditional code from the compiled preview.

Add a config block and set the materialization. Then the conditional code is included in the preview:

Remove the config block, and move to the models.yml file:

The code is no longer present in the preview:

However if I run a dbt compile and then view the output, the conditional code is present:

Hope this helps!