Created cluster should be be a valid parallel::cluster object

[hongooi73]

Since we steal the makeCluster and stopCluster function names, we should also duplicate their existing functionality in toto.

cl <- doAzureParallel::makeCluster("misc/batchConfig.json")
parallel::clusterEvalQ(cl, 42)
# Error in checkCluster(cl) : not a valid cluster

This would add a bit of extra flexibility in what you can do with the cluster; see ?parallel::clusterApply for what the parallel package provides.

[paselem]

Hi @Hong-Revo I think what you can/should do in this case is use rAzureBatch directly (rather than the doAzureParallel wrapper). We need to update the API version for it (working on that already and should have it out very soon) but it will essentially let you create a cluster of nodes and expose public IPs and Ports through which you can communicate to each node directly. Setting up the cluster should be pretty quick and straight forward.

The one point I'm not sure about at the moment is if/how to get data onto your nodes (do you want to use Azure Storage Blobs?). If that is a non-issue I think we will be able to support this scenario reasonably easily in a sibling project to doAzureParallel.