exq

exq copied to clipboard

Is there a good way to do (or plans to add such functionality) so? Recurring Jobs is really useful concept for some maintenance jobs like periodic cleanup

maybe even cron-style, e.g. https://github.com/resque/resque-scheduler#scheduled-jobs-recurring-jobs

P.S. probably https://github.com/c-rack/quantum-elixir is the solution

There is nothing built in to exq, but you can definitely use a cron style scheduler and then use the Enqueuer to queue jobs. :+1: to using quantum-elixir, I think that is probably the best approach.

I can also see there being an external package to make this integration easier (adding ability to declare jobs and parameters and maybe some UI display similar to resque-scheduler ), but unless there's a good reason, there's probably not a need to include this functionality in the exq project. Anyways, let me know what you end up doing and whether there ends up being a good reason to integrate this into exq and we can discuss.

One more thing possibly to consider as well is dealing with multiple nodes, making sure that the cron scheduling only runs on one node out of the group.

wow quantum-elixir is great!

@umurgdk did you use it with Exq for scheduling jobs, how did it work out?

@akira umm im just trying to learn elixir and experimenting https://github.com/umurgdk/elixir_resources. But for me it works perfectly :) Quantum just triggers the module like:

config :quantum, cron: [
  "* * * * *": {"ElixirResources.Workers.CronJobs", :look_videos},
  "* * * * *": {"ElixirResources.Workers.CronJobs", :look_rss},
]

For example :look_rss does;

def look_rss do
    IO.puts "Loading all resources from RssResource"

    Enum.each Repo.all(RssResource), fn(rss_resource) ->
        IO.puts "Loading rss resource: #{rss_resource.url}"
        Exq.enqueue :exq, "aggregator", "ElixirResources.Workers.RSSWorker", [rss_resource.url]
    end
end

We didn't tested yet with hundreds of resources. But we thought it might me useful to enqueue a job for each rss url and let the exq handle these jobs.

@umurgdk looks good, will check the project out! Let me know if you run into any issues.

@akira cool thanks :)

Closing for now as quantum-elixir can handle this. Will reopen if anyone brings up any needed changes for exq.

I'm going to reopen this issue - will be open to any implementations for exq. I was actually thinking it would be nice to tie this in the exq UI as well.

BTW, we had a similar requirement and created https://github.com/activesphere/exq-scheduler

Closing this, we have exq-scheduler and the support for UI has been merged to exq_ui master