snowflakedb
SNOW-2409156: Add write_parquet function equivalent to write_arrow
What is the current behavior?
There is no native function in snowpark-python or the snowflake-python-connector to ingest a folder of parquet files in a performant way to Snowflake. The closest is Session.write_arrow which already has all required logic.
What is the desired behavior?
Add Session.write_parquet(folder_with_parquet_files, ...) that reuses the logic from write_arrow. Refactor write_arrow to first create the parquet files and than calls Session.write_parquet
How would this improve snowflake-snowpark-python?
Many customers already have parquet files written by other engines that need to be quickly ingested into Snowflake. Avoiding the arrow step adds the most performant ingestion method to snowpark-python.
References, Other Background
Thank you for the PR @nicornk . Team will review and approve accordingly.
@nicornk thank you for the PR. Do you think the existing session.file.put(), then session.read.parquet() work for you?
