spark-celery
spark-celery copied to clipboard
gregbaker
Helper to allow Python Celery tasks to do work in a Spark job.
now if i run tasks quickly like below, no waiting , one by another: async_exec.delay("a");async_exec.delay("b"); it will raise a Error: > File "/usr/hdp/current/spark2-client/python/pyspark/sql/functions.py", line 1957, in wrapper > return udf_obj(*args)...
Dear all, I try to use spark-celery in my application, when i do like the code below, I get an error "AttributeError: 'SparkCeleryApp' object has no attribute 'sc'". Could you...
Dear all, I am using spark-celery in my application. I am calling my query-System (where I need to access broadcasted big files) from spark-celery. I want to broadcast (using spark-context)...