fix(optimizer)!: query schema directly when type annotation fails for processing UNNEST source
Issue: https://fivetran.atlassian.net/browse/RD-1066808
Problem: In BigQuery, when you UNNEST an ARRAY<STRUCT<...>>, the struct fields can be referenced as unqualified columns (e.g., WHERE type = 'x'). This worked in simple queries but failed in correlated subqueries because sqlglot's type annotation couldn't resolve columns from outer scopes, so qualify_columns didn't know what struct fields to expose.
Solution: Add a fallback mechanism in qualify_columns.py that queries the schema directly when type annotation fails. When processing an UNNEST source, if the expression isn't typed, traverse parent scopes to find the column's type definition in the schema (handling both table sources and CTEs), then extract and expose the struct field names as available columns in that scope.
can you fix the type annotation instead?
Yeah, I second that suggestion in favor of avoiding making this more complicated than necessary. Keen to understand more if that is tricky to do; let me know and we can hop in a call and chat about this.