Anwar Nunez-Elizalde

Hi, thanks for submitting the bug report! Unfortunately, I am not familiar with AWS EMR at all. My guess is that the code is being executed by a user who...

Hello, Great that you solved the issue by creating a `/home/.config` directory (though it's odd that the path is not `/home/$USER/.config`). With respect to the `PicklingError` you ran into: Is...

p.s. I found this: https://stackoverflow.com/questions/40674544/apache-spark-reads-for-s3-cant-pickle-thread-lock-objects It suggests that the issue is the serializing of the `cottoncandy.interface`. It suggests using `mapPartitions` instead of `flatMap`.

👍

Just had a quick look b/c it was on my mind. This should be pretty straightforward (see below). This should really only be allowed for uncompressed arrays and GZIP compressed...