Can't seem to get remote workers to pick up any work.

[TravisLeithVeloqx]

I am running redis on Windows from WSL Ubuntu 18.04. I can connect to redis, from a Windows remote machine (my slave machine) and I can even see the jobs via doRedis::jobs() I can start a job from R running on Windows (not WSL) on the same machine running redis (via WSL) and I can then add workers from a different R process on the same machine. The job I am testing with comes from the manual. However, when I try to startLocalWorkers() from a different machine, targeting the right queue, nothing happens. Also, when I try redisWroker(...) from the slave machine, it tells me it is waiting for jobs, I see no indication in CPU usage that something is happening, then after a while it says Normal worker exit.

Any idea how to get this to work?

[TravisLeithVeloqx]

2nd test:

I ran the following code on a machine that does not have redis

registerDoRedis('jobs', host = "192.168.128.6")
foreach(j=1:10,.combine=sum, .multicombine=TRUE) %dopar%
  4*sum((runif(1000000) ^ 2 + runif(1000000) ^ 2) < 1) / 10000000

The host is the machine that is running redis. I can also see that the queue exists via doRedis::jobs(). On a the same machine I then run redisWorker('jobs', host = "192.168.128.6")

and I immediately get the following output

> redisWorker('jobs', host = "192.168.128.6")
Waiting for doRedis jobs.
Error in `parent.env<-`(`*tmp*`, value = globalenv()) : 
  use of NULL environment is defunct

[bwlewis]

OK, thanks that is a good error to go on.

Can you tell me which R version this is?

On 12/18/19, TravisLeithVeloqx [email protected] wrote:

2nd test:

I ran the following code on a machine that does not have redis

registerDoRedis('jobs', host = "192.168.128.6")
foreach(j=1:10,.combine=sum, .multicombine=TRUE) %dopar%
  4*sum((runif(1000000) ^ 2 + runif(1000000) ^ 2) < 1) / 10000000

The host is the machine that is running redis. I can also see that the queue exists via doRedis::jobs(). On a the same machine I then run redisWorker('jobs', host = "192.168.128.6")

and I immediately get the following output

> redisWorker('jobs', host = "192.168.128.6")
Waiting for doRedis jobs.
Error in `parent.env<-`(`*tmp*`, value = globalenv()) :
  use of NULL environment is defunct

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/bwlewis/doRedis/issues/52#issuecomment-567058993

[TravisLeithVeloqx]

I am using R 3.6.1 on both machines.

[bwlewis]

Strangely, I can't reproduce this yet on Linux and Mac hosts with R 3.6.1 or R-devel (R 4). I am actively working on doRedis this month and trying to reproduce this. I'll try to get my hands on a Windows machine to test on this week.

Your error is happening here: https://github.com/bwlewis/doRedis/blob/master/R/redisWorker.R#L36 That line of code needs to eventually be changed anyway. But would still like to understand what's going on here.

[bwlewis]

I think I figured this out, but have not checked on a Windows box yet. I'm using basename(tempfile("")) here: https://github.com/bwlewis/doRedis/blob/master/R/doRedis.R#L348 as a way to uniquely name the work queue (along with process ID and host name). I think that might be where the *tmp* came from above in your failed example, and that is causing a problem.

I'll revise this.

[bwlewis]

Sorry this too quite a while. I think that the commit 3103d3395463e322e01112f20306b7c92142259d might fix this, along with a number of other environment-related issues.

This one is not yet the next CRAN version, which is still under development. That version will depend on the Redux package instead of rredis. But for now you can install from GitHub.