Fix: MSSQL add N char
Summary by CodeRabbit
-
New Features
- Added connectors for Redshift, Postgres, SQL Server, and Canner.
- DuckDB can attach additional databases and list/attach files from S3 or local storage.
- Query results now return as Apache Arrow tables for improved performance and compatibility.
- Automatic handling of Decimal and UUID types for consistent outputs.
-
Bug Fixes
- Unified, clearer error messages across connectors; safer dry-run behavior.
- More robust connection closing to prevent crashes or double-closing.
- MSSQL correctly handles Unicode string literals.
- Preserved timeout semantics while improving invalid SQL detection.
Walkthrough
Adds multiple connectors (Redshift, Postgres, MSSQL, Canner), changes query return type to PyArrow tables, introduces unified error handling, implements close semantics, enhances DuckDB behaviors, and adds utilities for UUID/Decimal post-processing and PostgreSQL type-name retrieval in ibis-server/app/model/connector.py.
Changes
| Cohort / File(s) | Summary of Changes |
|---|---|
Connector additionsibis-server/app/model/connector.py |
Introduces RedshiftConnector, PostgresConnector, MSSqlConnector, and CannerConnector with query/dry_run implementations and close methods. |
API return type updatesibis-server/app/model/connector.py |
Updates Connector.query and SimpleConnector.query to return pa.Table instead of pandas DataFrame. |
Unified error handlingibis-server/app/model/connector.py |
Wraps backend exceptions into WrenError with ErrorCode/ErrorPhase; adjusts mappings (e.g., timeouts vs INVALID_SQL) across query and dry_run. |
PyArrow post-processingibis-server/app/model/connector.py |
Adds _handle_pyarrow_unsupported_type, _cast_uuid_columns, _round_decimal_columns in SimpleConnector to cast UUIDs to string and round Decimals to a fixed scale. |
Close semantics and safetyibis-server/app/model/connector.py |
Adds close() methods to multiple connectors with safeguards to avoid double-closing and segfaults. |
DuckDB enhancementsibis-server/app/model/connector.py |
Enables attaching additional DuckDB databases, listing/attaching files via S3/Local FS interfaces, and improves dry_run/close safety. |
PostgreSQL type helperibis-server/app/model/connector.py |
Adds _get_pg_type_names(connection) to retrieve PostgreSQL OID-to-type-name mappings. |
Sequence Diagram(s)
sequenceDiagram
autonumber
actor Client
participant Connector as SimpleConnector
participant Ibis as Ibis Backend
participant Arrow as PyArrow
Note over Connector,Ibis: Query execution with PyArrow result
Client->>Connector: query(sql, limit)
Connector->>Ibis: compile/execute(sql, limit)
Ibis-->>Connector: result (backend table)
rect rgb(245,245,255)
Note right of Connector: Post-process unsupported types
Connector->>Connector: _handle_pyarrow_unsupported_type()
Connector->>Connector: _cast_uuid_columns()
Connector->>Connector: _round_decimal_columns(scale=9)
end
Connector->>Arrow: to_arrow()
Arrow-->>Connector: pa.Table
Connector-->>Client: pa.Table
sequenceDiagram
autonumber
actor Client
participant Conn as AnyConnector
participant Backend as Driver/DB
participant Err as WrenError
Note over Conn,Backend: Unified error handling for query/dry_run
Client->>Conn: dry_run(sql) / query(sql)
Conn->>Backend: execute / explain
alt Success
Backend-->>Conn: OK / result
Conn-->>Client: Response
else Backend raises (Trino/ClickHouse/psycopg/...)
Backend-->>Conn: Exception
rect rgb(255,245,245)
Note right of Conn: Map to ErrorCode + ErrorPhase
Conn->>Err: wrap(exception)
end
Err-->>Client: WrenError
end
sequenceDiagram
autonumber
actor Client
participant RS as RedshiftConnector
participant Psy as psycopg Cursor
participant Arrow as PyArrow
Note over RS,Psy: Redshift autocommit + Python-level cursor
Client->>RS: query(sql, limit)
RS->>Psy: execute(sql)
Psy-->>RS: rows / cursor data
RS->>Arrow: to_arrow()
Arrow-->>RS: pa.Table
RS-->>Client: pa.Table
Client->>RS: close()
RS-->>Client: closed
Estimated code review effort
🎯 4 (Complex) | ⏱️ ~60 minutes
Possibly related PRs
- Canner/wren-engine#1247 — Both touch DuckDBConnector: attach/list helpers and query return/signature changes align with this PR.
- Canner/wren-engine#1207 — Shares
_get_pg_type_nameshelper introduction and cursor/close handling in the same module.
Suggested labels
ibis, python
Suggested reviewers
- douenergy
- goldmedal
Poem
(\/) ( ••)✧ hop! New bridges built to data streams, UUIDs tamed, and Decimal gleams. Errors corralled, one burrowed voice— Arrow feathers our swift choice. Close the tunnel—quiet, poised.
Pre-merge checks and finishing touches
❌ Failed checks (1 warning)
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage | ⚠️ Warning | Docstring coverage is 17.50% which is insufficient. The required threshold is 80.00%. | You can run `@coderabbitai generate docstrings` to improve docstring coverage. |
âś… Passed checks (2 passed)
| Check name | Status | Explanation |
|---|---|---|
| Description Check | âś… Passed | Check skipped - CodeRabbit's high-level summary is enabled. |
| Title Check | ✅ Passed | The title mentions adding an N prefix for MSSQL Unicode literals, which corresponds to a valid subset of the changes. However, it does not reflect the primary objective of this pull request, which is the broad expansion of multi-backend support—including new Redshift, Postgres, MSSQL, and Canner connectors—along with enhanced PyArrow handling, unified error wrapping, and robust closing logic. |
✨ Finishing touches
- [ ] 📝 Generate Docstrings
đź§Ş Generate unit tests
- [ ] Create PR with unit tests
- [ ] Post copyable unit tests in a comment
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands and usage tips.
@ansutung, could you rebase on the latest main and add the description for what you want to propose?
It may be a duplicate of https://github.com/Canner/wren-engine/pull/1338