Gil Vernik

icon indicating a website link https://gvernik.blogspot.com

icon location Reality is what refuses to go away once you stop believing in it

Results 29 comments of Gil Vernik

Cannot use append mode when writing spark dataframe on Watson Studio

@charles2588 thanks for reporting this. In general append + object storage is usually a bad idea, no matter which connector you use. I will review the issue you observed to...

HTTP Client Version Warning

@theLegendary1 I don't think i see this warning...I tested with Spark 3.2, Hadoop 3.2..

spark strucutred stream with cos source use stocator fail by connection time out from pool

@student2028 have you tried to change default configuration values? https://github.com/CODAIT/stocator#cos-connector-optional-configuration-tuning there pool size, connection timeouts,etc.

Use cloudpathlib for the storage API?

@abourramouss @cisaacstern @TomNicholas I understand there might be different API for Storage layer. We implemented our own in Lithops, you suggest another way. What is rationale for your suggestion? Do...