Feat: Ensure compatible table schema by adding missing columns and expanding column sizes

[GovindHede]

Summary

This pull request ensures that the table schema is compatible with the stream's schema by adding any missing columns and expanding column sizes if necessary.

Changes Made

1. Implemented _ensure_compatible_table_schema method to:
   • Add any missing columns to the table.
   • Ensure column sizes are compatible with the stream's schema.
2. Added helper methods:
   • _get_stream_schema: Retrieve the schema for the specified stream.
   • _get_table_schema: Retrieve the schema of the specified table.
   • _is_size_expansion_needed: Check if the column size needs to be expanded.
   • _generate_alter_column_statement: Generate an ALTER TABLE statement for expanding column size.
   • _execute_alterations: Execute a list of ALTER TABLE statements.

Issue Reference

- 321

Summary by CodeRabbit

• New Features

  • Enhanced schema compatibility checks to automatically adjust column sizes and add missing columns in tables.
  • Introduced methods to retrieve stream and table schemas dynamically, ensuring seamless updates without manual intervention.

• Bug Fixes

  • Resolved potential issues with column size adjustments that were previously unhandled.

These changes improve the system's adaptability to schema changes, enhancing overall reliability.