spark-redshift
spark-redshift copied to clipboard
redshfit writer
I tried to run below command to save data in temp location in s3 in csv format. But it still stores data in avro format. Can you please tell me where it went wrong. I am using pyspark 2.3.2 and spark redshift connector 2.11 dataFrame.write.format("com.databricks.spark.redshift").option("url", url).option("tempdir", tempDir).option("tempFormat", "CSV").option("dbtable","test_csv1").option("aws_iam_role", iam).mode('overwrite').save()
Originally posted by @vishooo in https://github.com/databricks/spark-redshift/issue_comments#issuecomment-451120653