Unable to create a pool that has many long names

[jcoyne]

If the number of queues is large or the queues have long names, I'm unable to instantiate a pool. It seems like there's a 1022 character limit.

See https://github.com/ruby/psych/issues/395

[nevans]

I've been meaning to add an alternate config syntax, with the current config syntax as a shorthand notation. It's never really come up as an issue for me though.

I'd personally like something like the following:

workers:
  # worker types can be named, which can be useful if you are defining multiple "pools"
  # all worker types are added, as is, to the "default" pool
  urgent:
    count: 5
    queues:
      - "foo:urgent"
      - "bar:urgent"
  slow:
    count: 2
    queues:
      - "foo:slow"
      - "bar:slow"
  normal:
    count: 10
    queues:
      - "foo:urgent"
      - "bar:urgent"
      - "foo:normal"
      - "bar:normal"

  # can still use old shorthand style, but the worker type will have no name
  # "anonymous" workers defined here can only be used in the "default" pool
  "foo,bar,baz,quux": 30

  foo_only:
     # if count is 0, it won't be in "default" pool
     count: 0
     queues: ["foo:urgent", "foo:normal", "foo:slow"]
  bar_only:
     # if count is unspecified, should it default to 1 or 0? 
     queues: ["bar:urgent", "bar:normal", "bar:slow"]

# Can choose a specific "pool" via command line switch or ENV var.
# This would make it easy to use a single config file for multiple sets of workers, which would
# be useful if different workers need to run on specific hosts, or with a specific environment 
pools:
   override_counts:
      - urgent: 10
      - normal: 100
      - slow # but can also use the counts from above
   slow_only: ["slow"]
   foo:
     foo_only: 10 # must override, since they were set to zero by default
   bar:
     bar_only: 10

But, as I said, this isn't a priority for me, so... PRs are welcome (for some or all of the above) :)

[nevans]

To deal with your issue though, you ought to be able to make and use your own very simple config loader, that can easily workaround the character limit:

https://github.com/nevans/resque-pool#custom-configuration-loader

[jcoyne]

@nevans thanks for the tips. 👍