minion icon indicating copy to clipboard operation
minion copied to clipboard

[Feature Request] Minion Backend list_jobs: using args as search filter

Open atoomic opened this issue 4 years ago • 5 comments

list_jobs allow to filter on @$options{qw(ids notes queues states tasks)} but this would be a great addition to be able to filter on args

Simple use case:

  • avoiding duplicate pending jobs in the queue [using a guard]

Workaround using a note... which can be used in the search. Otherwise will have to look for every inactive jobs...

 my $mykey = join '_', @$args;
 my $guard_key = $mykey;
 my $guard = $minion->guard( $guard_key, $timeout );
 my $search = $minion->backend->list_jobs( 0, 1, { 
    notes => [ $mykey ], # workaround
    states => [ 'inactive' ] 
    } );

# already a pending job for this same task, avoid adding one... 
return if $search && $search->{total}; 
my $id = $minion->enqueue( something => $args, { notes => { $mykey => 1 } } );

atoomic avatar Jan 09 '20 17:01 atoomic

Problem is performance, since args are arbitrary data structures i don't see how they could ever be searched efficiently.

kraih avatar Jan 09 '20 17:01 kraih

It doesn't have to be the $args itself Same $mykey as in the given example may have a better placement than notes key

Many use cases would take advantage of identifying an instance of a task performed onto a given object

While reducing a queue of items, knowing that a job has been already queued for a given item->id would be handy

arhak avatar May 04 '20 03:05 arhak

You can already work around this if you know that you need to search it by putting the id you're interested in also into notes. I'm fairly certain that's what we concluded last time I brought this up on IRC

rabbiveesh avatar May 04 '20 10:05 rabbiveesh

Yes, i believe we already have the best possible solution for this problem implemented with indexed notes keys. There is nothing more for us to do.

kraih avatar Jun 01 '20 14:06 kraih

Having a slow search option with an args like '%foo%' search is still a possibility, since it would be rather useful for the admin ui.

kraih avatar Aug 03 '20 16:08 kraih