PyAirbyte icon indicating copy to clipboard operation
PyAirbyte copied to clipboard

Performance: Explore alternative to fetching all schemas in base.py

Open bindipankhudi opened this issue 1 year ago • 0 comments

Since we already know the schema name, we could potentially check whether it exists rather than fetching all schemas and then comparing:

def _get_schemas_list(
        self,
        database_name: str | None = None,
    ) -> list[str]:
        """Return a list of all tables in the database."""
        inspector: Inspector = sqlalchemy.inspect(self.get_sql_engine())
        database_name = database_name or self.database_name
        found_schemas = inspector.get_schema_names()
        return [
            found_schema.split(".")[-1].strip('"')
            for found_schema in found_schemas
            if "." not in found_schema
            or (found_schema.split(".")[0].lower().strip('"') == database_name.lower())
        ]

bindipankhudi avatar Feb 20 '24 18:02 bindipankhudi