Keivan Soleimani

Results 4 comments of Keivan Soleimani

Thanks for your response, There was no any error at runtime, I tested my code with "**apache/spark-py**" image (https://hub.docker.com/r/apache/spark-py) and the result was **correct** and the parquet file was created...

I tested the python code for saving dataframe to **json** format, but the result was the same problem as I mentioned before : ```python from pyspark.sql import SparkSession spark =...

with scala shell (spark-shell), everything is ok. ``` val df = spark.read.csv("csv/file.csv") df.write.mode("overwrite").format("json").save("file_name.json") ``` ![jsonScala](https://github.com/bitnami/containers/assets/53554624/fc93c482-7650-487f-9d91-b6d8689a2cc2) ![jsonScalaFile](https://github.com/bitnami/containers/assets/53554624/e56fc4b7-335c-4430-8a93-2e4995612ce7) but with pyspark and spark-submit python code file not found !

I tested the java code for saving dataframe to json format, but the result was the same problem as I mentioned before : ![Javacsvread](https://github.com/bitnami/containers/assets/53554624/b12e87b0-f475-424b-b52f-6b88833dfdc3) ![JavacsvreadSchema](https://github.com/bitnami/containers/assets/53554624/fe4f3a09-60d0-4062-91c2-cf5bd7196c7e) ![JavacsvreadNoFile](https://github.com/bitnami/containers/assets/53554624/80ad7f86-0487-4fa1-a10d-1f20f2335ffe) ```java package arka; import...