Allow user to customize table schema while creating

[utkarsharma2]

Please describe the feature you'd like to see

We are not using Table's columns attribute when we create a table. Currently, we rely on load_file_to_table which internally uses pandas to auto-detect the schema while creating a table. There is no way where users can supply schema of their own.

Describe the solution you'd like

Table objects have field columns which is supposed to have column data if the user has passed in columns using this field we should create a table by using the database.create_table() method and instruct pandas/libs(Bigquery, Snowflake) to append to the table rather than creating a new one.

Acceptance Criteria

• [ ] All checks and tests in the CI should pass
• [ ] Unit tests (90% code coverage or more, once available)
• [ ] Integration tests (if the feature relates to a new database or external service)
• [ ] Example DAG
• [ ] Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
• [ ] Exception handling in case of errors
• [ ] Logging (are we exposing useful information to the user? e.g. source and destination)
• [ ] Improve the documentation (README, Sphinx, and any other relevant)
• [ ] How to use Guide for the feature (example)

[tatiana]

@utkarsharma2 as we discussed, the BaseDatabase has a create_table method which already uses the table.columns, which are in SQLAlchemy format :)

So I believe it is just a matter of using that. What I think is missing in the BQ implementation of load_file is to use/support this feature - potentially both on native and on non-native implementations. I'm handling this for Snowflake ATM, and I asked Daniel to take care of the Postgres one.

[sunank200]

This is already implemented as discussed with @utkarsharma2