Embucket
DBT GitLab triggers a status code 200 response from the embucket
DBT GitLab is failing due to an issue with the embucket and becomes stuck while attempting to reconnect:
WARNING:snowflake.connector.vendored.urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<snowflake.connector.vendored.urllib3.connection.HTTPConnection object at 0x139326ee0>: Failed to establish a new connection: [Errno 61] Connection refused')': /queries/v1/query-request?requestId=cfd1b2e4-75a0-4a53-8d69-aea5742f895c&clientStartTime=1752780646061&retryCount=1&retryReason=None
Previously the model elastic_billing_data_transfer_by_deployment_source just fail but the whole run was not interrupted:
Database Error in model elastic_billing_data_transfer_by_deployment_source (models/sources/elastic_billing/elastic_billing_data_transfer_by_deployment_source.sql) 000200: DataFusion error: Schema error: No field named storage_and_transfer.value. Did you mean 'rate_value'?. compiled code at target/run/gitlab_snowflake/models/sources/elastic_billing/elastic_billing_data_transfer_by_deployment_source.sql
Now it make embucket abort and and gets stuck in a loop while attempting to connect to the database.
Error on the embucket side:
`{"timestamp":"2025-07-17T19:29:24.070399Z","level":"ERROR","fields":{"error":"DataFusion error: SQL error: ParserError("Expected: column name or constraint definition, found: 0 at Line: 2, Column: 54")"},"target":"core_executor::query"}
{"timestamp":"2025-07-17T19:29:24.123685Z","level":"ERROR","fields":{"error":"DataFusion error: SQL error: ParserError("Expected: column name or constraint definition, found: 0 at Line: 2, Column: 54")"},"target":"core_executor::service"}
{"timestamp":"2025-07-17T19:29:24.123773Z","level":"ERROR","fields":{"error":"DataFusion error: SQL error: ParserError("Expected: column name or constraint definition, found: 0 at Line: 2, Column: 54")"},"target":"api_snowflake_rest::handlers"}
{"timestamp":"2025-07-17T19:29:24.124037Z","level":"INFO","fields":{"message":"close","time.busy":"143µs","time.idle":"31.2µs"},"target":"api_snowflake_rest::error","span":{"error":"SQL error: ParserError("Expected: column name or constraint definition, found: 0 at Line: 2, Column: 54")","error_stack_trace":"DataFusion error: SQL error: ParserError("Expected: column name or constraint definition, found: 0 at Line: 2, Column: 54")\n0:
thread 'tokio-runtime-worker' has overflowed its stack fatal runtime error: stack overflow zsh: abort ./target/debug/embucketd`
SQL could be found here:
https://dbt.gitlabdata.com/#!/model/model.gitlab_snowflake.elastic_billing_data_transfer_by_deployment_source?g_v=1&g_i=%2Belastic_billing_data_transfer_by_deployment_source%2B
Seems like an issue comes from this part:
INNER JOIN LATERAL FLATTEN(input => PARSE_JSON(dims.value), outer => TRUE, mode => 'ARRAY') AS storage_and_transfer
Could be because before for each request by snowflake connector we would get a new session and a new DedicatedExecutor. So if one would fail because of the model (retry and then abort), it would just waddle on, on the other ones (DedicatedExecutor). However, this is just a symptom.
