jialin

Results 17 comments of jialin

Example error messages are like: ``` jialin.liu@mlx-engine-dp4coo9bxanc2u-0 ➜ /mlx/data00 python test_spark.py INFO:root:Connected to pre-provisioned cluster:10.192.191.15:32192 ERROR:ray.util.client.worker:ray::SparkOp.run() (pid=2550, ip=192.168.77.197) print(self._jdf.showString(n, 20, vertical)) File "/home/ray/anaconda3/lib/python3.7/site-packages/py4j/java_gateway.py", line 1305, in __call__ answer, self.gateway_client, self.target_id,...

Very often, it just doesn't print anything.

not sure the issue is from raydp or ray, sometimes i also see the following errors: ``` (raylet) terminate called after throwing an instance of 'std::bad_alloc' (raylet) what(): std::bad_alloc ```

after running the code on ray head with out using ray.util.connect, and not use ray.remote to submit raydp op, it seems my code has some issue in accessing hdfs, ```...

It might be this: AppMasterJavaBridge.scala ``` def startUpAppMaster(extra_cp: String): Unit = { if (instance == null) { // init ray, we should set the config by java properties Ray.init() instance...

By looking at Ray's [Java API](https://github.com/ray-project/ray/blob/master/java/runtime/src/main/resources/ray.default.conf), it looks like we need to use ray.conf to modify the address

I tried to set the address like this: ``` def startUpAppMaster(extra_cp: String): Unit = { if (instance == null) { System.setProperty("ray.address", "auto") // init ray, we should set the config...