John Cherian comments

Results 13 comments of


                                            John Cherian

Update Dockerfile

thank you @paulorangeljr and @ashishgupta-in let me review this

Hudi Job on Lambda - Directory /var/task/metastore_db cannot be created.

Have tried adding HUDI configuration "hoodie.write.markers.type":"direct", # It's not advisable to use this configuration. Working on workaround without using this config. "hoodie.embed.timeline.server":"false" # It's not advisable to use this configuration....

Hudi Job on Lambda - Directory /var/task/metastore_db cannot be created.

Closing it because of no response

reading csv from s3 throwing this error using (sample-spark-script-csv-to-csv.py) file

@saiteja2420 we are using S3a://

reading csv from s3 throwing this error using (sample-spark-script-csv-to-csv.py) file

@saiteja2420 if it csv to csv , then hadoop-aws.jar file version might be the problem, have you changed any version in the Dockerfile?

reading csv from s3 throwing this error using (sample-spark-script-csv-to-csv.py) file

No activity

Amazon Linux 2023 version works has dependency issues

Comparing the .toml files for the master branch and version 3.3.0 of PySpark, the notable differences are in the tool.black section. For the master branch, the required version of Black...

Amazon Linux 2023 version works has dependency issues

this is more of an FYI

Spark configuration for local deployment

Closing this