CSV file format while writing data to redsfhit.

[vishooo]

Hi I am trying to use below code dataFrame.write.format("com.databricks.spark.redshift").option("url", url).option("tempdir", tempDir).option("tempFormat", "CSV").option("dbtable","test_csv1").option("aws_iam_role", iam).mode('overwrite').save()

But it is still storing data in avro format in s3 temp location. I am suing spark-redshift 2.11 connector

Originally posted by @vishooo in https://github.com/databricks/spark-redshift/issue_comments#issuecomment-451117579

[agnarok]

I'm having the same problem but using dataframe.read

[ballcap231]

I believe you need spark 3.x in order to save tempdir in CSV format. Spark really should, at least, give an error when you try to specify an invalid tempdir file format. See this: https://github.com/databricks/spark-redshift/issues/308