goworker
goworker copied to clipboard
benmanns
goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers.
There were major changes in [see history](https://github.com/vitessio/vitess/commits/65cbfaf4c57b60c2af3ae6d573365c1b0f39f11c/go/pools). The changes on May 25th, 2019 broken [goworker](https://github.com/benmanns/goworker/blob/master/redis.go#L35) package. Added required parameter with default value ( default value 0 as defined in struct...
``` [vagrant@localhost gadmin]$ go get github.com/benmanns/goworker go get: github.com/youtube/vitess@none updating to github.com/youtube/[email protected]: parsing go.mod: module declares its path as: vitess.io/vitess but was required as: github.com/youtube/vitess ```
This logic is ported from the Ruby. It allows each worker to heartbeat Redis so if everything is killed instead of keeping them on the DB it'll try to prun...
Replaced redigo client with goredis one. @benmanns what is the significance of `GetConn` and `PutConn` ? I've removed them because of redis lib change, but they can be recovered if...
# github.com/benmanns/goworker /data/gopath/src/github.com/benmanns/goworker/redis.go:29:9: cannot use func literal (type func() (pools.Resource, error)) as type pools.Factory in return argument /data/gopath/src/github.com/benmanns/goworker/redis.go:35:30: not enough arguments in call to pools.NewResourcePool have (pools.Factory, int, int, time.Duration)...
I'm new to Go, and am trying to figure out things. One thing which made goworker a bit hard to use was that the worker get an `args ... interface{}`...
If the connection to redis goes down (due to redis restart, netsplit, ...) the library does not recover, but only logs the error: ``` 2014-03-10 19:07:23,682 DEBG 'my-worker' stdout output:...
It can happen that workers get stuck silently. We were using `node-resque` worker before, which handled this scenario very well. With `goworker`, jobs just keep shown as running in `resque-web`....
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 489 chaosii 20 0 1110252 17164 12524 S 54.2 0.1 1:38.45 background_work
