sfguide-data-engineering-with-snowpark-python
sfguide-data-engineering-with-snowpark-python copied to clipboard
Incorrect logic to map snowcli to snowpark connection params
Inside utils/snowpark_utils.py
THIS WILL NOT WORK AS THE KEYS BETWEEN SNOWCLI AND SNOWPARK ARE DIFFERENT!!
# Convert snowsql connection variable names to snowcli ones
# session_config_dict = {
# snowsql_to_snowpark_config_mapping[k]: v.strip('"')
# for k, v in session_config.items()
# }
WE NEED TO IETRATE OVER BOTH DICTIONARIES AT THE SAME TIME
session_config_dict = {}
for (k1,v1), (k2,v2) in zip(snowsql_to_snowpark_config_mapping.items(), session_config.items()):
session_config_dict[k1] = v2.strip('"')
Also, inside steps/02_load_raw.py
perhaps make this more foolproof
# current_dir = os.getcwd()
current_dir = os.path.dirname(os.path.abspath(__file__))